-- Logs begin at Thu 2022-08-25 01:51:11 UTC, end at Thu 2022-08-25 03:32:45 UTC. -- Aug 25 02:46:04.929319 dv-cinder-779578-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 25 02:46:04.939760 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 25 02:46:04 2022] *** Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: nodename: dv-cinder-779578-6-iscsi Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: machine: x86_64 Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: clock source: unix Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: pcre jit disabled Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: detected number of CPU cores: 4 Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: current working directory: / Aug 25 02:46:04.941827 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: detected binary path: /usr/bin/uwsgi-core Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: your processes number limit is 47680 Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: your memory page size is 4096 bytes Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: detected max file descriptor number: 2048 Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: lock engine: pthread robust mutexes Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: thunder lock: enabled Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 25 02:46:04.942421 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 25 02:46:04.983221 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: Python main interpreter initialized at 0x55db7b9549c0 Aug 25 02:46:04.983221 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: python threads support enabled Aug 25 02:46:04.983221 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: your server socket listen backlog is limited to 100 connections Aug 25 02:46:04.983221 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: your mercy for graceful operations on workers is 90 seconds Aug 25 02:46:04.983552 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: mapped 403077 bytes (393 KB) for 2 cores Aug 25 02:46:04.983650 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: *** Operational MODE: preforking *** Aug 25 02:46:04.983924 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: *** uWSGI is running in multiple interpreter mode *** Aug 25 02:46:04.983924 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: spawned uWSGI master process (pid: 81773) Aug 25 02:46:04.984111 dv-cinder-779578-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 25 02:46:04.987756 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: spawned uWSGI worker 1 (pid: 81785, cores: 1) Aug 25 02:46:04.987756 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: spawned uWSGI worker 2 (pid: 81786, cores: 1) Aug 25 02:46:04.987756 dv-cinder-779578-6-iscsi devstack@keystone.service[81773]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 25 02:46:09.376714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:46:09.376714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: from cryptography.utils import int_from_bytes Aug 25 02:46:09.378334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:46:09.378334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: from cryptography.utils import int_from_bytes Aug 25 02:46:09.378481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:46:09.378481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: from cryptography.utils import int_from_bytes Aug 25 02:46:09.378481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 25 02:46:09.378481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: from cryptography.utils import int_from_bytes Aug 25 02:46:09.429334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 25 02:46:09.429515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 25 02:46:09.429604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 25 02:46:09.429699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 25 02:46:09.429801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] command line args: [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 25 02:46:09.429887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] command line args: [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 25 02:46:09.429967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 25 02:46:09.430046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 25 02:46:09.430125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ================================================================================ {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 25 02:46:09.430204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ================================================================================ {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 25 02:46:09.430374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] admin_token = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] admin_token = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] config_dir = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] config_dir = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.430975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] config_source = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] config_source = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] debug = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] debug = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.431926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.432988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_config_append = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_config_append = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_dir = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_dir = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_file = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_file = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.433880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.434979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.435976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.436140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.436288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.449784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.449909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] publish_errors = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] publish_errors = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.450948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] standard_threads = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] standard_threads = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.451864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] use_journal = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] use_journal = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] use_json = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] use_json = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] use_stderr = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] use_stderr = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] use_syslog = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] use_syslog = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 25 02:46:09.452741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.452887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.452962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.453930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.454977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.455951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.456995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.457659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.458922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.459923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.460932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.461985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.462824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.463935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.464952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.465973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.466940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.467982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.468951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.469966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.external = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.external = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.470922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.password = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.password = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] auth.token = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] auth.token = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.471973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.472445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.474731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.474817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.474881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.474941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.475962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476227 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.476899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.477930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.478963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.479943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.480939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.481985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.482993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.483961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484644 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.484955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.485974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.486879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.487990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.488856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.489961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.490938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.491883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.492853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.493977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.494984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.495921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.496888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.497890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] role.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.498886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] role.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] role.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] role.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.499995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.500838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.501977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.502940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.503911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.504869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.505889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.506875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.507954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.508958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.509925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.510957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.511955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.connection = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.512115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.514809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.connection = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.514939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.515987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.516995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 25 02:46:09.517662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81786) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 25 02:46:09.517754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81785) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 25 02:46:09.547333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.547561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.547788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.547994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.548252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.548484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.556837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.557506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.585902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.586129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.586238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.586365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.651623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.652442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.701525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.701746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.703585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.703823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.861672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.861867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.861983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.862905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.891902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.892800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.904025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.904580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.917585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.917819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.918959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.919058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81786) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.919152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.919314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81785) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 25 02:46:09.943224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.943433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:46:09.943688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.943781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.943903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:46:09.943989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:46:09.944244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.944330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.944463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 25 02:46:09.944548 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.944742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.945031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.945112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.945381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.945561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.945831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.954836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.955923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.956237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.962823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.963464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.965836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.966121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.966400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.966484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.966680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.966765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.966962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.967844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.968989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.969264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.993147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.993320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.994714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.994822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.995000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.995112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:09.999673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.999836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:09.999965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.000129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.000280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.000465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.006812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.007094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.007227 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.007519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.011183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.011493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.011708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.011808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.012966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.013153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.013236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.013424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.013695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.031335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.031480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.031622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.031766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.035624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.035714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.039556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.039658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.039855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.039941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.044091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.044258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.047276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.047376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.050742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.051000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.051159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.051306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.051448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.059757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.059841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.059918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.059983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.060051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.060140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.060257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.068541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.068826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.068915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.069089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.069173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.069349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.069434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.069646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.076393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.076669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.077917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.078192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.078288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.078473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.080893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.081176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.081264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.081465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.081564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.081758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.081895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.082025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.082157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.082316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.094140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.094419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.094598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.094729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.094876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.095875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.096007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.096322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.111966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.112442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.115411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.115697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.118339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.118625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.119804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.120058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.121309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.121594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.121722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.121885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.122026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.122208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.128375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.128769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.130151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.130470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.133078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.133378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.133515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.133654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.133806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.133987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.134210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.134350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.141912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.142214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.142404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.142490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.142701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.142986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.149904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81786) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.150191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.150471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.150647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81785) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 25 02:46:10.150733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.150942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.151893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.152928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81786) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.153734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.154011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.154316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81785) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 25 02:46:10.188554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55db7b9549c0 pid: 81786 (default app) Aug 25 02:46:10.189946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55db7b9549c0 pid: 81785 (default app) Aug 25 02:46:10.192318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b17424-5391-490c-931e-894ed995d85a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:10.192678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b17424-5391-490c-931e-894ed995d85a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:10.192954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b17424-5391-490c-931e-894ed995d85a None None] PATH_INFO: `/v3` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:10.197174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1/1] 10.222.0.75 () {50 vars in 791 bytes} [Thu Aug 25 02:46:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:17.713950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:17.714488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:17.714488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:17.718979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1/2] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:46:17.724690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:17.725090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:17.725243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:17.797018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: INFO dbcounter [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] Registered counter for database keystone Aug 25 02:46:17.819663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG oslo_db.sqlalchemy.engines [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81786) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 25 02:46:17.825836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writer thread running {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 25 02:46:18.009381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG passlib.handlers.bcrypt [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81786) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 25 02:46:18.009728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG passlib.handlers.bcrypt [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81786) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 25 02:46:18.064845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:18.066457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:18.196887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 2/3] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 25 02:46:18.669820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:18.741794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: INFO dbcounter [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] Registered counter for database keystone Aug 25 02:46:18.761706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG oslo_db.sqlalchemy.engines [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81785) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 25 02:46:18.767757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writer thread running {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 25 02:46:18.889273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78f3c0-42da-4cd6-ba50-6b5a8b011b63 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:18.893547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:18.894164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:18.894446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:18.894728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:18.903609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:18.956453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING keystone.common.rbac_enforcer.enforcer [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 25 02:46:18.956797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f962d8-668f-4835-8b99-2ffbd177aaf9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:18.967011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 2/4] 10.222.0.75 () {58 vars in 1324 bytes} [Thu Aug 25 02:46:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 02:46:18.972986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:18.987702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cee31-a09b-4999-9b61-5879959312f6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:18.993399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:18.994504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:18.994788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:18.995062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] PATH_INFO: `/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:18.996668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] RBAC: Authorizing `identity:get_project(project_id=eabb40ce2ddb40b4bea318792fc09f8e)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:19.050934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING keystone.common.rbac_enforcer.enforcer [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 25 02:46:19.051499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545919-b07f-4e7d-91d0-d18f9141b644 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:19.056046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 3/5] 10.222.0.75 () {56 vars in 1207 bytes} [Thu Aug 25 02:46:18 2022] GET /identity/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:20.977147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e867e-98bc-444b-9019-0de2081255b1 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:20.977634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e867e-98bc-444b-9019-0de2081255b1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:20.977716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e867e-98bc-444b-9019-0de2081255b1 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:20.981851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 3/6] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:20.987087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:20.987485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:20.987641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:21.007911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:21.009523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:21.017389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 4/7] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 25 02:46:21.283515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e867e-98bc-444b-9019-0de2081255b1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:21.296104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e867e-98bc-444b-9019-0de2081255b1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:21.300494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:21.301091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:21.301371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:21.301677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:21.309191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:21.310945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1908a-9b10-454d-8179-e36bcd49bfeb admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:21.319731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 4/8] 10.222.0.75 () {58 vars in 1324 bytes} [Thu Aug 25 02:46:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 02:46:21.325097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:21.334603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dd310-6f94-4751-9791-975070fd5b49 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:21.338990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:21.339618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:21.339901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:21.340204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:21.341519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:21.343351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f823b3-69d1-4532-a3d2-18cc21945d1d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:21.344923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 5/9] 10.222.0.75 () {56 vars in 1198 bytes} [Thu Aug 25 02:46:21 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:23.343483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4675371-e5f8-49c4-9796-7c735ad0833e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.344019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4675371-e5f8-49c4-9796-7c735ad0833e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.344019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4675371-e5f8-49c4-9796-7c735ad0833e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.345464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 5/10] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:23.350965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.351430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.351533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.369755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186474-1e4a-4c52-8f43-2d474bc2d34f None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.370044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186474-1e4a-4c52-8f43-2d474bc2d34f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.370326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13186474-1e4a-4c52-8f43-2d474bc2d34f None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.371643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 6/11] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:46:23.371953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:23.373262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:23.378970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.379913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.380410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.422553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG passlib.handlers.bcrypt [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81785) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 25 02:46:23.423050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG passlib.handlers.bcrypt [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81785) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 25 02:46:23.455877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 6/12] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:23.469793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:23.471403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:23.569370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 7/13] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:23.730734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:23.756347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:23.838633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08419e45-4cee-4be9-8603-e8c389315c31 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:23.839011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:23.874414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:23.875027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.875310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.875586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.946289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0c7e-51e9-4093-b636-c50cd12d8771 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:23.948693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:23.953990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:23.955483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:23.979804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:23.984095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:23.985174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:23.985458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:23.985760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:23.998626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:24.007580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:24.008756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:24.009720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 8/14] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:46:24.014580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.059568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 7/15] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:24.065123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.116807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154b1b2-11b7-4f18-9bdd-3ae90afc99a1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.150815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.151424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.151705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.151979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.152681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:24.152990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.154441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.164110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace1224-ec3a-4cf5-9bdc-d8ed909c45be None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.166521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 9/16] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:24.172033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.187294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:24.199134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:24.199465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.206280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.206280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.206280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.211205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.212722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:46:24.230991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:24.232079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:46:24.233125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:46:24.233125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:46:24.233125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:24.233125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 8/17] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:46:24.237819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.275395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9fc46-40e1-462f-864c-0b27b91f92f6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.310468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.311105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.311387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.311664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.313194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.314653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.316265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 10/18] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:24.322539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.342923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534d685-1cea-4a20-bec5-6e7bce99ea5d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.382417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.383064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.383304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.383578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.384261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:24.384569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.385966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.403186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 9/19] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:24.419168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:24.427012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf3d56-3515-4cdb-956f-086ca2bf0a14 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.465512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.466168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.466447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.466720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.467897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.469406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5200-2d38-4d2a-a0bf-b4b77a7ac9eb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.471071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 11/20] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:24.479251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:24.517244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:24.587281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9ff68-1355-4b53-8f50-58ebf097de3a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:24.622896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:24.623517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:24.623859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:24.624120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:24.624890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.domain.id=default, user.id=ba566fd127094ca6936cf4d2fd42c8e6) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:24.625205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:24.626678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f194d-ab94-4bf7-b663-93a9e8a62cf0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:24.634951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 10/21] 10.222.0.75 () {56 vars in 1322 bytes} [Thu Aug 25 02:46:24 2022] GET /identity/v3/role_assignments?user.id=ba566fd127094ca6936cf4d2fd42c8e6&scope.domain.id=default&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:26.615119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93619b-18ba-4f43-8dca-39884f4873fe None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:26.616126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93619b-18ba-4f43-8dca-39884f4873fe None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:26.616651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93619b-18ba-4f43-8dca-39884f4873fe None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:26.618298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 12/22] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:26.623948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f148-2bbd-44da-9eb7-087eaf753fd2 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:26.624869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f148-2bbd-44da-9eb7-087eaf753fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:26.625413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f148-2bbd-44da-9eb7-087eaf753fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:26.653190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-67e9f148-2bbd-44da-9eb7-087eaf753fd2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:26.654789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-67e9f148-2bbd-44da-9eb7-087eaf753fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:26.750445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 11/23] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:27.044871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a647-06ad-42eb-87a9-3669c2a72b07 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.045566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a647-06ad-42eb-87a9-3669c2a72b07 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.046085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a647-06ad-42eb-87a9-3669c2a72b07 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.047644 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 13/24] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:27.053799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.054341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.054823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.081414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:27.083524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:27.113369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:27.164791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 12/25] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:27.248106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548a647-06ad-42eb-87a9-3669c2a72b07 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:27.389172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548a647-06ad-42eb-87a9-3669c2a72b07 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:27.443362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:27.444426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.444954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.445536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.473263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:27.475084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-cdbacedc-8f0a-4fe6-b5d8-53b8a11d5ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:46:27.493898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:27.494783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:46:27.495580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:46:27.495580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:46:27.495580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:27.497346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 14/26] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:27.503212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:27.528835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acac2-80c4-47fb-80d6-5f607084d057 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.531375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acac2-80c4-47fb-80d6-5f607084d057 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.531877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2acac2-80c4-47fb-80d6-5f607084d057 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.533450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 15/27] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:27.567604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5628-4d2f-42e9-a3d0-f5d286f29e69 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.568272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5628-4d2f-42e9-a3d0-f5d286f29e69 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.568756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5628-4d2f-42e9-a3d0-f5d286f29e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.606858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-869a5628-4d2f-42e9-a3d0-f5d286f29e69 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:27.609625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-869a5628-4d2f-42e9-a3d0-f5d286f29e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:27.636524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:27.686951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c8d32-595a-4ea1-a698-d418f8fe8734 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:27.725287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 16/28] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:27.727522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3288-4a8b-423e-88a2-b420019b43c5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.728011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3288-4a8b-423e-88a2-b420019b43c5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.728537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3288-4a8b-423e-88a2-b420019b43c5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.729991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 17/29] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:27.734063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:27.734063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.734063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.734063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.734380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:27.734380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:27.734505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f6bab-ad0c-454a-af16-4a270150016a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:27.735493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3288-4a8b-423e-88a2-b420019b43c5 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:27.755833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 13/30] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:27.759360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.759687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.759972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.802295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:27.803506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:27.835215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:27.864094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3288-4a8b-423e-88a2-b420019b43c5 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:27.896312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 14/31] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:27.897576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:27.904264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:27.904932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:27.905219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:27.905557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:27.906292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:27.907938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cdba9-b5fa-4cd0-974e-cb7ce533c9b4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:28.001590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c86b6-2d3f-483e-8c43-e558c2d86c5b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:28.045798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:28.046451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.046738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.047018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.047682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:28.049502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa94b-ee41-4158-8ddd-205784f3b773 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:28.149247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 18/32] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Aug 25 02:46:27 2022] POST /identity/v3/roles => generated 225 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:28.151155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a16a6-8e90-49e7-95f1-fb2c46cd9da4 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.151654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a16a6-8e90-49e7-95f1-fb2c46cd9da4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.152137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a16a6-8e90-49e7-95f1-fb2c46cd9da4 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.153678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 19/33] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:28.154586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:28.154978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a16a6-8e90-49e7-95f1-fb2c46cd9da4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.159332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:28.300503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 15/34] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:46:27 2022] POST /identity/v3/projects => generated 301 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:28.303694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230682d7-34ef-455d-a2d3-0366155445df None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.304394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230682d7-34ef-455d-a2d3-0366155445df None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.305227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230682d7-34ef-455d-a2d3-0366155445df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.321661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a16a6-8e90-49e7-95f1-fb2c46cd9da4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:28.327744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-230682d7-34ef-455d-a2d3-0366155445df None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:28.329768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-230682d7-34ef-455d-a2d3-0366155445df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:28.376029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:28.376712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.376997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.377275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.377967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:28.380295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381c0c5-c0cd-4dcb-b9c3-52ff999dcaa9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:28.423228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 16/35] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:28.423870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230682d7-34ef-455d-a2d3-0366155445df None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.473185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 20/36] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Aug 25 02:46:28 2022] POST /identity/v3/roles => generated 219 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:28.475690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1746-b335-42a3-9059-565778010567 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.477258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1746-b335-42a3-9059-565778010567 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.477712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e1746-b335-42a3-9059-565778010567 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.479270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 21/37] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:28.485118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.487327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.487937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.514378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:28.515979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:28.554654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230682d7-34ef-455d-a2d3-0366155445df None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:28.596416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:28.607453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 22/38] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:28.702820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:28.709998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:28.711079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.711589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.712115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.716816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.729152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:28.736865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:28.737792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:28.739186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 17/39] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:28.745116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.832574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca424495-457e-4791-b3cc-af8d75250a25 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:28.856561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5804-5bad-4191-a516-ec6a80c44979 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:28.869434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:28.870046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.870328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.870604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.871262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:28.872688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:28.892334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:28.892958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:28.895120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:28.895120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:28.895120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:28.895120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:28.896134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:28.908009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 18/40] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:28.908909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.947354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 23/41] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Aug 25 02:46:28 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:28.948387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:28.969014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:29.023534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de579eea-c662-45f2-9db8-dbc2bbe21f0f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:29.024721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:29.063155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:29.063818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:29.064107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:29.064459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:29.065670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:29.067746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:29.069462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 19/42] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:29.075220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:29.082983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202973f-3f83-4cc3-b2ba-d2222eacf2d9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:29.121034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:29.121801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:29.122084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:29.122367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:29.124888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:29.126369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:29.128133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 24/43] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:28 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:29.134568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:29.188981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac87dfb-0644-4b48-a9fb-19a4f130066d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:29.209227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:29.223436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:29.226145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:29.226145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:29.226145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:29.226145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:29.226703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2c8a-05e6-488e-be51-12f4a0b16447 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:29.239651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e893a0-c9e6-455b-82e2-0d8210b61f30 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:29.261398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:29.273634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:29.274256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:29.274572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:29.274849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:29.275960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:29.277428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:29.279022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 25/44] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:29.284639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:29.329556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 20/45] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:46:29 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:46:29.389255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f41634-6101-4ec7-8b3d-03493cad79f5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:29.426532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:29.427233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:29.427511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:29.427812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] PATH_INFO: `/v3/domains/default/users/ba566fd127094ca6936cf4d2fd42c8e6/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:29.429850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ba566fd127094ca6936cf4d2fd42c8e6, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:29.431301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed583375-0926-454e-9cf6-7bccf19b2992 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:29.506084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 26/46] 10.222.0.75 () {58 vars in 1361 bytes} [Thu Aug 25 02:46:29 2022] PUT /identity/v3/domains/default/users/ba566fd127094ca6936cf4d2fd42c8e6/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:46:31.258090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de2265-37d3-4463-9c81-d6a5231e9051 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.258576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de2265-37d3-4463-9c81-d6a5231e9051 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.258576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de2265-37d3-4463-9c81-d6a5231e9051 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.259899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 21/47] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:31.265345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a402-0a94-4863-8f7f-c878ea23d1c1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.265761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a402-0a94-4863-8f7f-c878ea23d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.265912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a402-0a94-4863-8f7f-c878ea23d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.283943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c784a402-0a94-4863-8f7f-c878ea23d1c1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:31.285278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c784a402-0a94-4863-8f7f-c878ea23d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:31.331356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:31.352246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 27/48] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:31.380554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fd9e3-4d60-4bd7-98ff-4fc2bb1e948b None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.380839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fd9e3-4d60-4bd7-98ff-4fc2bb1e948b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.381113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fd9e3-4d60-4bd7-98ff-4fc2bb1e948b None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.382588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 22/49] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:31.387536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.387832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.388111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.406266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:31.407456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:31.472121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 28/50] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:31.622578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fd9e3-4d60-4bd7-98ff-4fc2bb1e948b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:31.724043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fd9e3-4d60-4bd7-98ff-4fc2bb1e948b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:31.746877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:31.758497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:31.759145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.759474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.759708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.760881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:31.762370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:31.764085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 23/51] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:31.769942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:31.812097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:31.847167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c79b41-6a90-42b8-9d85-ee5e3720c29c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:31.858299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:31.872509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434617ad-affc-4405-b849-78542b25ab85 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:31.881663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:31.882236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.882520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.882799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.892959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:31.898894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:31.899738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:31.904633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 29/52] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:46:31.905896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:31.907279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:31.907921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:31.908237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:31.908516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:31.909172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:31.911462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:32.017097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 24/53] 10.222.0.75 () {60 vars in 1140 bytes} [Thu Aug 25 02:46:31 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:46:32.017350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2c338-037f-4b46-ba8a-2ee1c8737ef4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:32.046488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:32.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:32.060586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 30/54] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:32.066512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:32.162880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:32.171472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25857-73e7-4a57-a3cb-77d9211ff186 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:32.210515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:32.213210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:32.213210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:32.213210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:32.213210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:32.214348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:32.215921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 25/55] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:32 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:32.221873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:32.257709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:32.333581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781604d5-5660-4968-9160-dab32028a4a7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:32.366914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:32.367562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:32.367850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:32.368129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:32.369281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:32.370737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2d54-aad7-4186-b976-627f588e61c1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:32.372344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 31/56] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:32.378093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:32.480085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec6a2f-9bfd-419f-a0c5-394f9b9b8630 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:32.513989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:32.514607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:32.514895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:32.515180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:32.515913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.domain.id=default, user.id=ba566fd127094ca6936cf4d2fd42c8e6) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:32.516258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:32.517729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26b79-7a73-4995-8e92-9af48a0504f6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:32.524384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 26/57] 10.222.0.75 () {56 vars in 1322 bytes} [Thu Aug 25 02:46:32 2022] GET /identity/v3/role_assignments?user.id=ba566fd127094ca6936cf4d2fd42c8e6&scope.domain.id=default&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:33.923467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b1328-9914-4d2d-9eba-d2248c19b7ba None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:33.923970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b1328-9914-4d2d-9eba-d2248c19b7ba None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:33.924053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b1328-9914-4d2d-9eba-d2248c19b7ba None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:33.925380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 32/58] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:33.931234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:33.931613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:33.931790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:33.950431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:33.951618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:33.969167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:34.017483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 27/59] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:34.294636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b1328-9914-4d2d-9eba-d2248c19b7ba None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:34.391294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:34.394943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b1328-9914-4d2d-9eba-d2248c19b7ba None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:34.428447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:34.429077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:34.429365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:34.429641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:34.430768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:34.432214 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80b461-7e87-4a6e-bfff-312b4b351288 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:34.433757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 33/60] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:34.439245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:34.534623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dd1b4-5f55-4e78-a383-0a16a8ab3492 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:34.568996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:34.569538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:34.569817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:34.570090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:34.570727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:34.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d7ea1-df02-46ae-838a-8ce6507c37f4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:34.671375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 28/61] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:46:34 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:37.672371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaef77-b1ee-4ab3-9332-83f37efeceb9 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.673167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaef77-b1ee-4ab3-9332-83f37efeceb9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.673167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaef77-b1ee-4ab3-9332-83f37efeceb9 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.674656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 34/62] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:37.685842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bacf7-3943-4045-9f4d-43323e696ca7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.686307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bacf7-3943-4045-9f4d-43323e696ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.686407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bacf7-3943-4045-9f4d-43323e696ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.717153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e38bacf7-3943-4045-9f4d-43323e696ca7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:37.718366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e38bacf7-3943-4045-9f4d-43323e696ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:37.721355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:37.809834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 29/63] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:37.863534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367b11-ef10-4498-8af9-2d2d02d5abc8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.863815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367b11-ef10-4498-8af9-2d2d02d5abc8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.864093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367b11-ef10-4498-8af9-2d2d02d5abc8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.865476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 35/64] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:37.874515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.874808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.875086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.902891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:37.904072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:37.912666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb940d-b774-46a0-8676-d5a29f19afc3 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.912951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb940d-b774-46a0-8676-d5a29f19afc3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.913226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb940d-b774-46a0-8676-d5a29f19afc3 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.916452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 36/65] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:37.920193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff6d1-fb68-41b6-a21c-06c0c590eda5 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:37.920193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff6d1-fb68-41b6-a21c-06c0c590eda5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:37.920422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cff6d1-fb68-41b6-a21c-06c0c590eda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:37.946578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-90cff6d1-fb68-41b6-a21c-06c0c590eda5 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:37.947785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-90cff6d1-fb68-41b6-a21c-06c0c590eda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:38.041694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 30/66] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:38.061221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 37/67] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:38.183978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.273960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:38.321551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc20a9-ab15-4222-b752-295dc54f11d8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.324915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c277-a16a-4221-bc81-3aabad07fdf1 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.325206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c277-a16a-4221-bc81-3aabad07fdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.325488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26c277-a16a-4221-bc81-3aabad07fdf1 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.326842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 38/68] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:46:38.337651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.337937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.338219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.361871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.362693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.363005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.363284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.370237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:38.371543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:38.376043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:38.383150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:38.384026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:38.384845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:38.386217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 31/69] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:38.387203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.451685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 39/70] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:38.453558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.494857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4273534-5d86-46f1-bd1c-70e4b2e739f1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.495273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:38.525345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:38.532280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.533247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.533545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.533964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.544953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-014fccd1-b571-4a10-bde1-d3c15950b71f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:38.551333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:38.552486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:38.553571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 32/71] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:38.557876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c3f06-aecf-4736-b59b-7821ac054df2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.558195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c3f06-aecf-4736-b59b-7821ac054df2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.558857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c3f06-aecf-4736-b59b-7821ac054df2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.560306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 33/72] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:38.564366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c3f06-aecf-4736-b59b-7821ac054df2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.584599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3b28b-e57d-406b-9a14-3fe0b0bc421b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.621027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.621640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.621987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.622229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.622860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:38.623167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:38.624584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:38.633927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 40/73] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:38.637961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.669553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c3f06-aecf-4736-b59b-7821ac054df2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.706222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.707561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.707848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.708126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.718874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0ea69fe7-b296-403c-91f0-327e5170b73f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:38.724314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:38.725232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:38.725989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 34/74] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:38.727165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.727428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.727715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.740119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:38.745058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:38.746278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:38.749160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16cf1-3606-45f0-98f8-e15d2ce1a927 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.783710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.784441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.785351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.785845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.786744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:38.787033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:38.788445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:38.793661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:38.796875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 41/75] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:38.797596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.818572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 35/76] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:38.819425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.917654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb2158-99f5-4b7b-994f-9ccc200e1e71 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.923342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea75b7-e9b1-4d39-9fe4-fd3285f37707 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:38.959513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.960058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.960384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.960658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:38.960812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.961225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:38.961568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:38.961853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:38.962017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:38.963295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:38.969797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 42/77] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:38.970657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.973893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:38.984420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:38.985541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:38.986534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 36/78] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:38.987343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:38.998837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.038258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.083764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f3416-fc81-4409-bcf9-7ba5c6b743b8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.105282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db10d52-3d9c-4e2f-9b14-351b1e778af9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.117524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.118131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.118411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.118681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.119367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:39.119671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.121130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.130208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 43/79] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.130853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.141906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.142561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.142944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.143267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.159295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.161160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.163459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 37/80] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:38 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.165005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.225876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.246834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91dc5-e58f-4518-ba4d-8dbbe31297f7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.273133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.278529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93173c88-16fd-4fa0-89a3-1d5109f59a8f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.286689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.287349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.287628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.287898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.324750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.325376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.325653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.325930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.326614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:39.326920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.328351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.337554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 38/81] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.338955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.379602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.381213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.461082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf701e7b-2d28-4f81-a9ae-995e32b5536f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.477138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.497221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 44/82] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:39.498086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.505999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.507034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.507541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.508048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.509995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.511861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.514139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 39/83] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.515461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.556596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.616003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2ab9-95fe-4832-a66c-543092e9b430 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.640019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde29b-cf08-4415-b038-d35a20d6992b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.654577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.655225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.655517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.655792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.658032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.659502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.661503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 45/84] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.662286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.683981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.684629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.684914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.685196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.717114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.760795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.762658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.775964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f7b06-1449-4a41-bea0-b3a206181c6b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:39.791202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:39.811893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:39.812574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:39.812935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:39.813340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:39.814527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:39.816511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:39.818077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 46/85] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:39.818979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:39.878128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 40/86] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:39.879850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.003919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3527961-b760-4174-aa9f-86afd73e2c38 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.030066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.030204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa677e-4a2b-4dd4-926b-3f64628ea1f7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.037896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.038515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.041675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.041675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.041675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.041675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.046736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.063979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.070062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 47/87] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:39 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:40.070831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.156843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 41/88] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:46:39 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:40.157676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.173055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7099d7-e0b7-4a6b-8f4e-12f0f88ea2ee None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.215993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.216696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.216985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.217277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.295614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.296419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e72a1-9fa9-4cb4-a05f-b6009564b1f2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.312670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.314320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.342415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.342637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.342905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.343243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.410327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.429037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.430552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.434211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 48/89] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:40.435098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.548649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.566335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfc113-9900-4fcf-b6c4-48e9ba348ae5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.566634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 42/90] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:40.567521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.612518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.614213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.614497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.614776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.616679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.618446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.620986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 49/91] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:40.622231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.664527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.688104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763ac7-af5a-4474-a176-c6fd59692356 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.735473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.736387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.736679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.736959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.737702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ba566fd127094ca6936cf4d2fd42c8e6) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:40.737879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c9c4-13fa-4440-b777-89ba148a7892 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.738567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.740315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.748058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 43/92] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/role_assignments?user.id=ba566fd127094ca6936cf4d2fd42c8e6&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:40.749454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.813758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.814396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.814732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.815017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.816856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.818411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.820217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 50/93] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:40.821033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.868120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:40.877237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d28d-1783-4d91-a33b-f8dbe8a76f6e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:40.941134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:40.941439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:40.941641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:40.941923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] PATH_INFO: `/v3/projects/50233c9c543441f8afd8d712cbeb111b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:40.944942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] RBAC: Authorizing `identity:get_project(project_id=50233c9c543441f8afd8d712cbeb111b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:40.946432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:40.948484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 44/94] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/projects/50233c9c543441f8afd8d712cbeb111b => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:40.949393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:40.977628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4bd0-79d1-4735-96c5-0264521fc735 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:41.005842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:41.017699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:41.018569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:41.019717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:41.020209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:41.021368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:41.022024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:41.023612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:41.033308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 51/95] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:41.034690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:41.132442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e20a89-be4b-4880-bdfd-f1a6bca42459 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:41.175824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:41.176545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:41.176837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:41.177119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:41.177956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:41.178273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:41.184167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9942-ec95-459c-836a-599ac4a2b0af None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:41.187347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 45/96] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:40 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:41.260218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:41.282153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df244-f44c-40af-b70f-8d5553134d12 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:41.339317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:41.339961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:41.340278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:41.340564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:41.341332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=50233c9c543441f8afd8d712cbeb111b, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:41.341650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:41.343135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad126d-e7a9-48e2-92be-4de4a37bbf40 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:41.349498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 52/97] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:41 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=50233c9c543441f8afd8d712cbeb111b&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:42.785412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de2f4-eb24-4548-9435-33415b137b26 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:42.785911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de2f4-eb24-4548-9435-33415b137b26 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:42.786221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de2f4-eb24-4548-9435-33415b137b26 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:42.787598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 46/98] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:42.802315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f9b99-c59c-4da6-8e58-7d76823bd53b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:42.802808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f9b99-c59c-4da6-8e58-7d76823bd53b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:42.802897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f9b99-c59c-4da6-8e58-7d76823bd53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:42.828521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3f4f9b99-c59c-4da6-8e58-7d76823bd53b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:42.829788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3f4f9b99-c59c-4da6-8e58-7d76823bd53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:42.914812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 53/99] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:43.045938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a48fb-2e15-49f9-bfa6-73322876c645 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.046234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a48fb-2e15-49f9-bfa6-73322876c645 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.046424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a48fb-2e15-49f9-bfa6-73322876c645 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.048043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 47/100] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:43.063169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.063461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.063743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.086796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:43.087994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:43.176057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 54/101] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:43.339864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a48fb-2e15-49f9-bfa6-73322876c645 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:43.438043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:43.461255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a48fb-2e15-49f9-bfa6-73322876c645 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:43.511532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:43.512193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.512486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.512767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.528978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:43.533838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:43.538828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:43.539695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:43.541153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 48/102] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:43.545765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:43.553277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:43.679741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba154d-b08d-4aa2-aa87-98a1beed5b94 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:43.727870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e20fce-e552-4826-9808-84190c4b1b06 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:43.746879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:43.747593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.747881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.748170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.749530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:43.751269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2428-0df2-4492-a0fc-cd5ed85b1b30 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:43.753939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 55/103] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:43.758233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05af2-e6cf-4a10-996f-5808849bc76e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.758555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05af2-e6cf-4a10-996f-5808849bc76e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.764482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05af2-e6cf-4a10-996f-5808849bc76e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.765972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 56/104] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:43.771378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05af2-e6cf-4a10-996f-5808849bc76e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:43.848170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:43.849130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.849130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.851943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.851943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:43.851943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:43.851943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0581b3-0be7-47a9-aa78-5cabf8be0303 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:43.862347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 49/105] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:43.865071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d970-1616-4477-8601-25784e42b160 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.865552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d970-1616-4477-8601-25784e42b160 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.866012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1d970-1616-4477-8601-25784e42b160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.885569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:43.914541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-40f1d970-1616-4477-8601-25784e42b160 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:43.916068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-40f1d970-1616-4477-8601-25784e42b160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:43.922456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:43.949651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05af2-e6cf-4a10-996f-5808849bc76e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:43.988806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:43.989624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:43.989871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:43.990144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:43.990798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:43.992373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f6b9-27d3-4be6-9af1-807b6a4d46df None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:44.049774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 50/106] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:44.051409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa7b5f-450c-4c2d-a80f-38258887fbc0 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.051906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa7b5f-450c-4c2d-a80f-38258887fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.052475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa7b5f-450c-4c2d-a80f-38258887fbc0 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.053948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 51/107] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:44.058849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa7b5f-450c-4c2d-a80f-38258887fbc0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.111989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 57/108] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:46:43 2022] POST /identity/v3/projects => generated 302 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:44.113524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eeeea-da7c-442e-b2d5-d9649a160d13 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.113814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eeeea-da7c-442e-b2d5-d9649a160d13 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.114183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eeeea-da7c-442e-b2d5-d9649a160d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.152017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-385eeeea-da7c-442e-b2d5-d9649a160d13 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:44.153304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-385eeeea-da7c-442e-b2d5-d9649a160d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:44.229441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa7b5f-450c-4c2d-a80f-38258887fbc0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.304261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.304444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 58/109] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:44.311202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665802f-cc61-49b3-96c3-c69c7f00c0ed None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.316463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665802f-cc61-49b3-96c3-c69c7f00c0ed None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.316463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665802f-cc61-49b3-96c3-c69c7f00c0ed None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.316621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:44.316621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.316621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.317069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 59/110] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:44.317310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] PATH_INFO: `/v3/users/33099e21b3d840fdb64d3a9152490845` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.326997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.332171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.332171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.366318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] RBAC: Authorizing `identity:get_user(user_id=33099e21b3d840fdb64d3a9152490845)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:44.368069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:44.369837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 52/111] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/users/33099e21b3d840fdb64d3a9152490845 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:44.373690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.375453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:44.378810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:44.478299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.500695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 60/112] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:44.500695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.527862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefbc3e-c67c-4927-937a-9704335d1b9a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.582799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:44.583454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.583739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.584015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.601693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:44.605232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0da0-3368-4f97-8fe6-0a5702eec376 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:44.606544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:44.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:44.608123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:44.616596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 53/113] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:44.622055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.644017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:44.644724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.645012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.645348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.688839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.710276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.721106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:44.722802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411514c-2c01-4ae7-992e-81d2f7056c65 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:44.754639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b1f59-d01f-46f1-904e-71970aebf0ed None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.795324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:44.796006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.796342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.796627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.797377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:44.798037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:44.799470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:44.808822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 54/114] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:44.809650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.853614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 61/115] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:44.855049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab33b-fb17-49ea-bddd-80c1ee390bdf None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.855347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab33b-fb17-49ea-bddd-80c1ee390bdf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.855628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab33b-fb17-49ea-bddd-80c1ee390bdf None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.857034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 62/116] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:44.857663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ab33b-fb17-49ea-bddd-80c1ee390bdf None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:44.921634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a5c-d6c6-43ba-a44a-c761a81365b7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.937266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.960245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:44.960886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:44.961171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:44.961449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:44.966151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:44.973957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:44.977654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ab33b-fb17-49ea-bddd-80c1ee390bdf None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:44.979571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:44.981576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:44.982379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:44.985331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 55/117] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:44.986207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.016708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.017276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.017581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.017864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.029494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-81fa1860-d998-4eee-91ad-ebaa1eef442d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:45.034336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:45.035705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 63/118] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:45.037212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.037939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.037939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.060617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:45.061828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:45.098334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b8ba-29f8-4a30-8c23-d94f34dbc25e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.135278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 64/119] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:45.135586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.135938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.136259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.136543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.137633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.137739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.139239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.140873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 56/120] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:44 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.142146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.189919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.229520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.249733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df6a8-3c65-4eba-b5a5-17365943a720 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.250042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef806-0184-4147-ad3a-f11a91b462c8 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.285018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.285557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.285841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.286124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.286783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:45.287095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.288597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.290100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.290983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.291400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.291809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.293081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.294766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.296688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 65/121] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.297014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 57/122] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.298031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.303168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.408534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279771f7-6826-43db-a836-da6a210ca17c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.410120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.411554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db07e-4aa5-4efd-892b-e45b534798c3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.445611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.446256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.446546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.446827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.447980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.449473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.449946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.450554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.450900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.451141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.451810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:45.452108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.453480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.456332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 58/123] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.457762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.469174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 66/124] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.470092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.483170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.569849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237a8bb-c483-4dde-a27f-bc120f35f6db None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.584116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c3fc-59d5-427b-9156-c44ef323c740 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.606648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.607398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.607622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.607975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.608720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=33099e21b3d840fdb64d3a9152490845) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:45.608977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.610496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.617983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 59/125] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/role_assignments?user.id=33099e21b3d840fdb64d3a9152490845&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.618687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.635288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.636200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.636311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.636515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.637772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.639407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.641131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 67/126] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.641956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.668693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.745611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42946b2f-1663-45d4-90e4-ca274cf24ff9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.758610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.782337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb793202-63d1-45b8-b7f1-029f5af25449 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.789307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.790236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.790813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.791291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.805898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:45.813497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:45.814382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:45.815163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:45.817643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 60/127] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:45.818327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.830933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.831990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.832541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.833018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ba566fd127094ca6936cf4d2fd42c8e6/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.835206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, user_id=ba566fd127094ca6936cf4d2fd42c8e6, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.836977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.907006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 68/128] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:45 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ba566fd127094ca6936cf4d2fd42c8e6/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:46:45.908777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:45.938649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:45.946480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb0428-ae04-4098-a524-10a5f5df143e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:45.987233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:45.988568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:45.988963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:45.989252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:45.990653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:45.992464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:45.994451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 61/129] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:45.995141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.039507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c20f45-51f4-4cbb-8196-c83dd9652153 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.120937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.121144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bba8e7-d1d6-46ea-ab11-71419cce89fc None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.121604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.121894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.122175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.123417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.124956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.126871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 69/130] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:46.132941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.154666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:46.167771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.170123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.170123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.170123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.170123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:46.170513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.172214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.185290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 62/131] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:46.186315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.223082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:46.311021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f177763-d42b-4be1-8481-41b65a7c9a03 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.313025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61650e-2d24-4dd7-a7be-cb735ddc601a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.351909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.352652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.353088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.353381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.355591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, user_id=ad5c7bc2451948c38fb9372c8a4cef9f, role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.357436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.380306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55788523-c576-4bed-bbb1-85426969919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.380876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55788523-c576-4bed-bbb1-85426969919c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.381158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55788523-c576-4bed-bbb1-85426969919c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.381490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55788523-c576-4bed-bbb1-85426969919c None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.382892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55788523-c576-4bed-bbb1-85426969919c None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.384429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55788523-c576-4bed-bbb1-85426969919c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.386063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 70/132] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:46 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:46.387038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55788523-c576-4bed-bbb1-85426969919c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.474958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:46.482473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 63/133] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:46 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/ee78bca73e4446e2b38a9e195677f064 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:46:46.483199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.505162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55788523-c576-4bed-bbb1-85426969919c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.546132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.546780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.547057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.547333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.553141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.554714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.556693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 71/134] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:46 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:46.563322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.607932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:46.626123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2837e-fba2-450f-9c92-c600919e9bfb None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.663201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.664284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.664778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.665253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.667519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, user_id=ad5c7bc2451948c38fb9372c8a4cef9f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.669282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa772a-8c50-4e03-a8d0-25d36b719543 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.741626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 64/135] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:46 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:46:46.742890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476d7a-c692-424c-9dcf-4f221a869ea0 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.743318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476d7a-c692-424c-9dcf-4f221a869ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.743660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476d7a-c692-424c-9dcf-4f221a869ea0 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.743760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae298bf-ae2f-4d27-b7d5-79f0d436fd1c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.745067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 65/136] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:46.751613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.751929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.753016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.777005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:46.778299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:46.787919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.788606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.788890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.789166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] PATH_INFO: `/v3/projects/50233c9c543441f8afd8d712cbeb111b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.791114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] RBAC: Authorizing `identity:get_project(project_id=50233c9c543441f8afd8d712cbeb111b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.793121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:46.795648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 72/137] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:46 2022] GET /identity/v3/projects/50233c9c543441f8afd8d712cbeb111b => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:46.802551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:46.817596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:46.943395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7bb1c-2b6f-4064-a4e2-4567345c85e5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:46.978910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 66/138] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:46.990255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:46.991162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:46.991716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:46.992136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] PATH_INFO: `/v3/projects/50233c9c543441f8afd8d712cbeb111b/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:46.994228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] RBAC: Authorizing `identity:create_grant(project_id=50233c9c543441f8afd8d712cbeb111b, user_id=ad5c7bc2451948c38fb9372c8a4cef9f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:46.995799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:47.088361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 73/139] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:46 2022] PUT /identity/v3/projects/50233c9c543441f8afd8d712cbeb111b/users/ad5c7bc2451948c38fb9372c8a4cef9f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:46:47.311874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:47.407775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:47.441394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5d97d-2992-43ed-affb-c590559a1b9f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:47.478302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:47.478952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:47.479234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:47.479511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:47.480659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:47.482135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf091c-d335-4803-8402-013e02bb9ca3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:47.483714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 67/140] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:47.490082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:47.579079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:47.616314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097016d-d2dd-402b-983f-1d9e895ccb34 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:47.657794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:47.658450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:47.658732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:47.659010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:47.660307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:47.661800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ad935-850f-4873-a56c-22e8f8964802 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:47.848893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 74/141] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:46:47 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:48.198207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3915c-c677-422e-82d2-aed10954c7b7 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:48.198705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3915c-c677-422e-82d2-aed10954c7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:48.198778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3915c-c677-422e-82d2-aed10954c7b7 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:48.200302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 68/142] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:48.205724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0886-5478-4a17-a7ba-1d614afe2db3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:48.206152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0886-5478-4a17-a7ba-1d614afe2db3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:48.207098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0886-5478-4a17-a7ba-1d614afe2db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:48.237153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-94ff0886-5478-4a17-a7ba-1d614afe2db3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:48.238465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-94ff0886-5478-4a17-a7ba-1d614afe2db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:48.314956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 75/143] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:48.457968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43404-64ca-4996-aa94-661d31dd5ba2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:48.458254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43404-64ca-4996-aa94-661d31dd5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:48.458447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43404-64ca-4996-aa94-661d31dd5ba2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:48.459942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 69/144] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:48.468531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:48.468531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:48.468531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:48.484493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:48.485995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:48.539114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:48.557098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 76/145] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:48.629569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43404-64ca-4996-aa94-661d31dd5ba2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:48.760837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43404-64ca-4996-aa94-661d31dd5ba2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:48.796913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:48.797805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:48.798095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:48.798365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:48.811070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:48.816637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:48.817488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:48.818274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 70/146] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:48.824803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:48.857722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:48.862021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:48.998138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd8d6d-e6f6-40ae-9312-ee7b7e9c5305 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.053688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07c92b-cd8c-43b3-8566-b9d2aaf4795d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.063527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.064213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.064488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.064765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.065476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:49.068461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.070323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.087124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 77/147] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:49.092513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.145878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.146976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.147464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.147938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.176763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:49.182136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:49.182956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:49.183748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:49.186203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 71/148] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:46:49.188131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:49.193077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.287401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94ad3-18c2-4fc0-ad87-7f3088582736 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.311701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:49.350819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.351473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.351757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.352033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] PATH_INFO: `/v3/users/33099e21b3d840fdb64d3a9152490845` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.356938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] RBAC: Authorizing `identity:get_user(user_id=33099e21b3d840fdb64d3a9152490845)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.358400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.360190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 78/149] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:49 2022] GET /identity/v3/users/33099e21b3d840fdb64d3a9152490845 => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:49.366558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.414198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2034-a6cd-4f01-b66f-57d6cffd9e52 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.493855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.494477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.494759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.495032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.495719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:49.496032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.497516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c771ab-354b-4397-9f92-e2fa51a8459b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.515447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 72/150] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:49.516801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587539-a532-4bc5-afb8-0f4e84badc46 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.517100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587539-a532-4bc5-afb8-0f4e84badc46 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.517381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587539-a532-4bc5-afb8-0f4e84badc46 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.518657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 73/151] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:49.522587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d587539-a532-4bc5-afb8-0f4e84badc46 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.590787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb115e0-3b4e-48f2-8c24-83fe29db2a4f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.629816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:49.652963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.653620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.653907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.654178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.656561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.656857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba2db-5d2e-4a68-831e-d4efb1cf906f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.661626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 79/152] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:49 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:49.661626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.661626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.661626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.692887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:49.694543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:49.714227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d587539-a532-4bc5-afb8-0f4e84badc46 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.772357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 80/153] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:49.773361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.776486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:49.812243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.812896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.813193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.813476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] PATH_INFO: `/v3/users/ba566fd127094ca6936cf4d2fd42c8e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.815048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] RBAC: Authorizing `identity:get_user(user_id=ba566fd127094ca6936cf4d2fd42c8e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.816840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.818705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 74/154] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:49 2022] GET /identity/v3/users/ba566fd127094ca6936cf4d2fd42c8e6 => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:49.824876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:49.916269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebeef7c-b6c4-4cf0-96b8-755da36a1efd None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:49.936528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:49.957467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:49.958011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:49.958379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:49.958662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/33099e21b3d840fdb64d3a9152490845/roles/d781c665e4d34e4e8014866ef91bdd52` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:49.961583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, user_id=33099e21b3d840fdb64d3a9152490845, role_id=d781c665e4d34e4e8014866ef91bdd52)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:49.963054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:49.993682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0c562-839e-4dad-8f63-a46bb282f75d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:50.030175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:50.030825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.031115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.031397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.060903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 81/155] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:49 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/users/33099e21b3d840fdb64d3a9152490845/roles/d781c665e4d34e4e8014866ef91bdd52 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:46:50.103730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:50.113528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:50.115088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:50.225019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 75/156] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:50.230866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:50.339990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:50.413068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967af3-c8fb-40c4-9297-e8effc039b0d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:50.467313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:50.467956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.468276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.468547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.469719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:50.471260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2197e-1fbc-4847-810b-7d62f5e96c50 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:50.473251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 82/157] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:50 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:50.474238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cc5e-62e2-463d-a0a8-c39fec538fcc None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.474518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cc5e-62e2-463d-a0a8-c39fec538fcc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.474787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13cc5e-62e2-463d-a0a8-c39fec538fcc None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.476040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 83/158] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:50.482603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a623a-5d5a-474e-85da-e14c86102f73 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.482603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a623a-5d5a-474e-85da-e14c86102f73 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.482603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a623a-5d5a-474e-85da-e14c86102f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.500855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c14a623a-5d5a-474e-85da-e14c86102f73 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:50.501885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:50.503653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c14a623a-5d5a-474e-85da-e14c86102f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:50.511715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775dcc-eb7e-4083-96d1-99a7a78997b8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:50.544635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:50.552364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:50.553298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.553788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.554265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.568132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:50.573511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:50.574467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:50.575253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:50.580602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 76/159] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:46:50.580602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:50.590281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 84/160] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:50.590793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6e77-ebed-4c73-a51c-403be3e96dc0 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.591144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6e77-ebed-4c73-a51c-403be3e96dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.591529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6e77-ebed-4c73-a51c-403be3e96dc0 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.592900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 85/161] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:50.593673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6e77-ebed-4c73-a51c-403be3e96dc0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:50.692696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2be67-b1df-4436-b80f-bb89d2ecfbd5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:50.749489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:50.750458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.751007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.751485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.752518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ba566fd127094ca6936cf4d2fd42c8e6) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:50.753144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:50.754874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86d62-66d9-48cc-9c02-a4acb5ecb925 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:50.761861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 77/162] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:50 2022] GET /identity/v3/role_assignments?user.id=ba566fd127094ca6936cf4d2fd42c8e6&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:50.763121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6e77-ebed-4c73-a51c-403be3e96dc0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:50.763677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.764186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.764661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.794284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:50.795949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:50.819843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:50.820555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:50.820836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:50.821162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:50.821822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:50.823051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:50.823567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:50.826015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:50.838936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 86/163] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:46:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:50.845804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:50.913377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 78/164] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:50.936583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.043305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f764b1-9e14-475d-95f4-a86807ba4109 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.091813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.092498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.092786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.093064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.094297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:51.095756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:51.097593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 87/165] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:50 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:51.103411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.170029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.249078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717f290-0cdd-479a-9ba7-4ded0abe88f7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.291695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.292399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.292677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.292948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.320038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084962-e7fb-4984-be33-1eef7faad023 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.320822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.324441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.386851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.387497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.387783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.388060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.402837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:51.406195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:51.408435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:51.413971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:51.415244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:51.416275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:51.420591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 88/166] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:51.426550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.556522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 79/167] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:51.557886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.594668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae02760-6d89-477c-8f8c-bc3224efc63f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.653765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.654426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.654715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.654991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.655669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:51.655993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:51.657407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:51.661298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.668271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 89/168] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:51.669219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.716534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.731282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e714-8686-4690-aaa1-5b9e8eb9cb68 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.777998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.779028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.779504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.780368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.801849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:51.807348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:51.808243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:51.809100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:51.811484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 80/169] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:51.812486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.904280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ffa92-18b6-46f8-b927-8c2e51462ca1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.952527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd224300-e03f-45b6-ac52-b56b0d28f647 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:51.960605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:51.983682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.984358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.984642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.985029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.986246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:51.988602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:51.989050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:51.989704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:51.989989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:51.990269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:51.990434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 90/170] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:51.992108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:51.993776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:51.995722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 81/171] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:51.996525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:51.997252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.105831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe1aac-dbd6-4742-a7b9-101fcd1ee7ea None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:52.141913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:52.142543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.142827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.143106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.144404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:52.144720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:52.146192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:52.152303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 82/172] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:52.153111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.180633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:52.196616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:52.200011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28988b7e-c1ac-4d34-9f0d-487e9b32d3c2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:52.294185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:52.294888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.295175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.295455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.296137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:52.296466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:52.298220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:52.316323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 91/173] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:52.322465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.324064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06827224-6e9d-407e-a05d-f8099c892031 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:52.425052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:52.426035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.426321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.426596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.494228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f8a5c-4ae9-4076-83be-b29c0be1c540 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:52.520594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:52.522652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f718a-317e-4ad4-a467-e678a383fb74 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:52.530357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:52.530991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.531268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.531537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] PATH_INFO: `/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.532786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] RBAC: Authorizing `identity:get_user(user_id=ad5c7bc2451948c38fb9372c8a4cef9f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:52.534871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b264fa-466c-45ab-a896-7ea3b0b85642 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:52.536986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 92/174] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:52 2022] GET /identity/v3/users/ad5c7bc2451948c38fb9372c8a4cef9f => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:52.538062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86523399-b96f-4a5c-8952-d93acefd44f8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.538343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86523399-b96f-4a5c-8952-d93acefd44f8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.538621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86523399-b96f-4a5c-8952-d93acefd44f8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.539835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 93/175] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:52.544716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86523399-b96f-4a5c-8952-d93acefd44f8 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.574338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:52.578465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:52.662755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86523399-b96f-4a5c-8952-d93acefd44f8 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:52.700497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 83/176] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:52.701033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:52.701843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.701974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:52.702337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.702631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:52.702745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.703186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:52.741212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:52.742494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:52.783634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:52.786475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:52.812699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:52.847287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 84/177] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:52.848171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.900965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 94/178] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:52.906964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:52.981741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.048420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdbf11-87be-49df-b547-acf7d7c22686 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.102377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.103027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.103310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.103584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.105497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:53.106981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430b3d8-f562-491b-8643-28e4b345811a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:53.108692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 85/179] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:52 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:53.109833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fd36-d751-4c20-b746-89cfc58e7459 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.110118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fd36-d751-4c20-b746-89cfc58e7459 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.110390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5fd36-d751-4c20-b746-89cfc58e7459 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.117628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 86/180] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:53.123333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fd36-d751-4c20-b746-89cfc58e7459 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.145529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa675f1-b2bd-4c5a-811c-5022b2e899ee None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.186899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.199898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.200728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.201012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.201290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] PATH_INFO: `/v3/projects/50233c9c543441f8afd8d712cbeb111b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.202439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] RBAC: Authorizing `identity:get_project(project_id=50233c9c543441f8afd8d712cbeb111b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:53.203899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a6b0f-e024-43cb-88e1-df9f6de776a3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:53.206061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 95/181] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:52 2022] GET /identity/v3/projects/50233c9c543441f8afd8d712cbeb111b => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:53.212452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.212743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.213024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.237955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:53.239275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:53.313762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5fd36-d751-4c20-b746-89cfc58e7459 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.320426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 96/182] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:53.321275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.379667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.380315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.380603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.380879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.381772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:53.382090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:53.383715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:53.399652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.408325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 87/183] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:53.409770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.468592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3735e-0ec4-474b-a607-f5688e99b112 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.520992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.536812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.540116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.540116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.540116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.540116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=50233c9c543441f8afd8d712cbeb111b, user.id=ad5c7bc2451948c38fb9372c8a4cef9f) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:53.540116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:53.540774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567a4f5-7588-4ff5-ae17-b5d40efd59f4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:53.547507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 97/184] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/role_assignments?user.id=ad5c7bc2451948c38fb9372c8a4cef9f&scope.project.id=50233c9c543441f8afd8d712cbeb111b&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:53.548976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3b3-5fe8-4cb1-ad98-08a740fe566f None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.549327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3b3-5fe8-4cb1-ad98-08a740fe566f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.549663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bc3b3-5fe8-4cb1-ad98-08a740fe566f None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.551464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 98/185] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:53.556362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a38bea-66b1-41ff-969d-73dd66c45d6a None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.556362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a38bea-66b1-41ff-969d-73dd66c45d6a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.556362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a38bea-66b1-41ff-969d-73dd66c45d6a None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.556362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 99/186] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:53.560816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.561145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.561382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.585901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:53.585901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:53.612200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43962ef1-db6f-4887-9968-e8984a221a94 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.652635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.653288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.653565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.653838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.666711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-1ea82bf0-e34e-476b-9d0a-e31b9841739e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:53.672402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:53.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:53.674206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 88/188] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:53.674949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 100/188] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:53.679606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.679880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.680056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.680478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.699066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:53.700289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:53.771228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.776361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 89/189] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:53.777186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.795593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce288-3400-4461-9dfc-000bfd05fd7e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.812136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:53.834553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.835176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.835454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.835709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.836479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:53.836733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:53.838160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:53.849252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 101/190] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:53.854859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:53.898741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e91dd-ff38-4950-b54a-858916b207ff None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.947382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:53.948370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:53.948924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:53.949205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:53.963584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:53.969800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:53.970716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:53.971517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:46:53.972203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35dde-9685-4101-a72a-15418c29f9ec None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:53.972298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 90/191] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:53.984938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.020016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.020838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.021111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.021381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.038035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.039493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.041486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 102/192] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:54.042164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.050609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.083813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.107002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707b45-e48d-4748-a763-7715a5ed1aa5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.141587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.142186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.142462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.142731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.143441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:54.144397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.145799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.153153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f9084-f510-4d48-a65e-af300a43836e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.153514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 91/193] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:46:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:54.154284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.188547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.189179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.189459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.189733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.191548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.193028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.194602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 103/194] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:54.195361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.255637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d5ce-a275-481d-9e17-1076a09841c4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.267854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.291264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.291863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.292139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.292432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.293524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c663dc2-ef86-4f77-ba3a-3c5b3709d692 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.295051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.328940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.329560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.329837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.330106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.340277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:54.344966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:54.345860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:54.346741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 104/195] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:54.348009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.366246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.367831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.451183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7c263-d0d6-4727-9cb4-dfa018e1571e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.467875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 92/196] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:54.468562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.481994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.484573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.485228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.485503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.485771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.486960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.489164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.490729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 105/197] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:54.491564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.524443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.566690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ab34-6ded-4c4d-9933-a6d5a883b37b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.593325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9f08-63a7-44ee-a313-9a6a01c79006 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.600433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.601045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.601327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.601602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.602244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.603709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.627151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.627692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.628799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.629145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.629814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:54.630121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.631988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.640229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 106/198] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:54.646365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.710475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 93/199] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:46:54 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:54.729253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.732781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.743549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9568-f6ce-4315-b095-7231bc5d6634 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.783062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.783770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.784623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.784946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.786382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.787863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.789836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 107/200] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:54.790479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.813115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:54.852314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede18b0-7259-4d1e-a5e4-f1f06ee3b1d7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.896405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.897246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.897668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.898094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.907808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b656383-e637-4896-b659-b0ec7403ecb2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:54.958372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:54.959142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:54.959420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:54.959690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] PATH_INFO: `/v3/users/33099e21b3d840fdb64d3a9152490845` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:54.960978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] RBAC: Authorizing `identity:get_user(user_id=33099e21b3d840fdb64d3a9152490845)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.962563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:54.964374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 108/201] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/users/33099e21b3d840fdb64d3a9152490845 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:54.964919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:54.996402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:54.998488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.002261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.082320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.105284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868042c4-79d7-42a0-a769-0d6943ac1410 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:55.120845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 94/202] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:55.121693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:55.153244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:55.154031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:55.154744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:55.155026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:55.156047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=1af36a254b9a41b4b9a1ac21a01adfe7) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:55.156631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:55.158803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.168131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 109/203] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:54 2022] GET /identity/v3/role_assignments?user.id=1af36a254b9a41b4b9a1ac21a01adfe7&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:55.169058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:55.258680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e26aa-fa3c-4e4c-a15a-2dbfc3e74b9c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:55.302773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:55.303404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:55.303683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:55.303955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:55.358972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.364902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.368967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885f8a-496b-43f3-9fe9-6ec85471a369 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:55.404742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:55.405750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:55.406255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:55.406722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:55.408132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:55.408265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:55.409933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.410156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.412450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 110/204] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:55 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:55.417998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:55.621234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 95/205] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:55.626867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443e338-28ee-49d9-9a87-4c40da66fb63 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:55.637551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:55.700283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:55.700283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:55.700283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:55.700283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:55.700663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=1af36a254b9a41b4b9a1ac21a01adfe7) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:55.700663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:55.700663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.706016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 111/206] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:55 2022] GET /identity/v3/role_assignments?user.id=1af36a254b9a41b4b9a1ac21a01adfe7&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:55.740261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.778803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833e7ae-f66c-4db5-8e2b-02695e6e1b00 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:55.825665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:55.826580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:55.827009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:55.827460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:55.828993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:55.830764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e77244-c719-4352-9058-fa450dbd2824 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:55.834955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 96/207] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:55 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:55.840299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:55.886122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:55.999944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca520-c685-4b33-8ec4-f60d88127d25 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:56.050199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:56.050821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:56.051101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:56.051378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:56.052124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb, user.id=33099e21b3d840fdb64d3a9152490845) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:56.052466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:56.053935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6310ee-88b8-479f-b18e-509aef39aabf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:56.064930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 112/208] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:55 2022] GET /identity/v3/role_assignments?user.id=33099e21b3d840fdb64d3a9152490845&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:57.039879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b920-24fb-4e61-9b99-e33428ce2b6a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.040419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b920-24fb-4e61-9b99-e33428ce2b6a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.040732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b920-24fb-4e61-9b99-e33428ce2b6a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.042471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 97/209] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:57.048065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc8ac-d354-4765-a383-e5c5fd52548b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.048556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc8ac-d354-4765-a383-e5c5fd52548b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.048743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fc8ac-d354-4765-a383-e5c5fd52548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.052582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece64df-30ce-45d7-8516-a8b03fc2c54d None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.052872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece64df-30ce-45d7-8516-a8b03fc2c54d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.053150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece64df-30ce-45d7-8516-a8b03fc2c54d None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.054795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 98/210] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:46:57.061158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde36b5-c069-4965-8a24-5a3c442eb8c6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.061441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde36b5-c069-4965-8a24-5a3c442eb8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.062260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde36b5-c069-4965-8a24-5a3c442eb8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.067798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3b8fc8ac-d354-4765-a383-e5c5fd52548b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:57.069319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3b8fc8ac-d354-4765-a383-e5c5fd52548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:57.084331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4bde36b5-c069-4965-8a24-5a3c442eb8c6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:57.085986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4bde36b5-c069-4965-8a24-5a3c442eb8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:57.155266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 113/211] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:57.167157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 99/212] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:46:57.278206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b29a-a455-433b-b508-4da9704ea7e2 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.278502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b29a-a455-433b-b508-4da9704ea7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.278665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0b29a-a455-433b-b508-4da9704ea7e2 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.280285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 114/213] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:57.285499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.285799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.286075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.305133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:57.306333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:57.368516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:57.376019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 100/214] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:57.435789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0b29a-a455-433b-b508-4da9704ea7e2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.452817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.460329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:57.556993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0b29a-a455-433b-b508-4da9704ea7e2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:57.583455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e2c97-befa-400e-97c5-f179dfddb290 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:57.602753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:57.603399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.603684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.603959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.620461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:57.620980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:57.621824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.622326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.622794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.624161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:57.625493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:57.626357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:57.627138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 115/215] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:57.630038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:57.633016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.637707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 101/216] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:57.639222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.711259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:57.716498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:57.739145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd08fa-7c40-4d15-a290-6cfb319d18ca None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:57.752673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1895-22e5-4d42-bd54-3555e0efa71b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:57.776443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:57.777150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.777410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.777599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.778262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:57.778572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:57.779951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:57.788894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 116/217] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:57.791929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:57.792573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.792857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.793239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.793778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:57.794016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.795230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:57.892935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 102/218] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:46:57 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:46:57.893842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:57.909467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25a199-860b-4324-89f6-e7e6cfe28b6b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:57.952032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:57.952706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.952987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.953256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.962250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:57.966116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-36b75953-8d29-4f81-8446-2e7c455ae0a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:57.971233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:57.972876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:57.973695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 117/219] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:57.979249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4f9db-52cc-432f-843a-c480fbdc3282 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:57.979800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4f9db-52cc-432f-843a-c480fbdc3282 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:57.980296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4f9db-52cc-432f-843a-c480fbdc3282 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:57.981874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 118/220] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:46:57.982694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4f9db-52cc-432f-843a-c480fbdc3282 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.025602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588628f-7976-4c1d-b507-40a8d82d7e7b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.050124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.068907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.069679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.069957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.070254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] PATH_INFO: `/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.091522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] RBAC: Authorizing `identity:get_user(user_id=b81bf7c64c7d4a5a84b6d3e0a9dd501b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.093123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c3077-db2f-4db6-8d6a-5d6acb74a12d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.094884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 103/221] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:58.096098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.102268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.102820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.116657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4f9db-52cc-432f-843a-c480fbdc3282 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.121542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:46:58.122740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:46:58.162026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.162353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.162592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.162863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.163541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:58.163846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.165961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.176593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 119/222] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:46:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:58.177805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.213716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 104/223] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:46:58.216962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.255476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.296867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a872b-c481-4e74-9e6f-72c074f13f82 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.344906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.345586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.345866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.346130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.347383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2529b3-f1e3-4e46-8ef8-5e29d11253a0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.360066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.419243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.419819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.420107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.420416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.424867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.426438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.428079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 105/224] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:58.438297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.470526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.472178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.538945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.587557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 120/225] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:46:58.588530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.603222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e773fa-dc1c-4d90-b37d-c504270a137d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.699277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95529ba-5243-4da6-bc4d-0b603dcc231f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.703326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.704286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.704785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.705231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.707614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.709831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.712857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 106/226] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:58.714688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.736282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.737755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.738080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.738353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.749551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:46:58.755248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:46:58.756129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:46:58.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 121/227] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:46:58.757673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.778740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.792461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:58.861059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b152d5-ac2a-4335-88fb-f21488acab08 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.873181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee32cb-be75-45ae-a188-0418dfde3d80 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:58.907920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.908468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:58.909678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.909916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:58.910087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.910311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/users/1af36a254b9a41b4b9a1ac21a01adfe7/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.910714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:58.911257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:58.912354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, user_id=1af36a254b9a41b4b9a1ac21a01adfe7, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.912803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:58.916064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.920699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:58.923246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 107/228] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:58.924329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:58.990081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 122/229] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:58 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/users/1af36a254b9a41b4b9a1ac21a01adfe7/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:46:58.990944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:59.053603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63348c-26ba-4945-82e8-66cfe757d35c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:59.073785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:59.090637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:59.104497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:59.105649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:59.106116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:59.106701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:59.108128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:59.108775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:59.110401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ae9a-42a2-4219-a62c-b5d4e62ae0ca None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:59.110681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:59.122759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 108/230] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:59.132259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:59.171754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:59.172643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:59.172915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:59.173188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:59.177350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=b81bf7c64c7d4a5a84b6d3e0a9dd501b) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:46:59.177665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:59.179147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:59.188963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 123/231] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:46:58 2022] GET /identity/v3/role_assignments?user.id=b81bf7c64c7d4a5a84b6d3e0a9dd501b&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:59.302791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b4880-1dc3-4974-9422-dfb477cd688f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:59.349326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:59.349987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:59.350268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:59.350546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:59.351707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:59.353207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:59.355154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 109/232] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:46:59 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:46:59.360902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:59.518015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7f22e-626c-4ecc-b01a-6b3aac52c58a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:59.569048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:59.569603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:59.569879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:59.570200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:59.571340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:59.572807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f93afe-1e7a-46a7-8213-2f9abba88ba5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:59.574711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 124/233] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:46:59 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:46:59.580188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:46:59.648231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:46:59.716931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a494e-5c5e-4509-9e43-457ea54e1485 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:46:59.753096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:46:59.753779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:46:59.754056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:46:59.754328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/users/1af36a254b9a41b4b9a1ac21a01adfe7/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:46:59.756636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, user_id=1af36a254b9a41b4b9a1ac21a01adfe7, role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:46:59.758654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f3aa-a2a8-4313-a576-72fae90efec9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:46:59.855419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 110/234] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:46:59 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/users/1af36a254b9a41b4b9a1ac21a01adfe7/roles/ee78bca73e4446e2b38a9e195677f064 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:00.239374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f509-d825-4540-bea3-5cf13ec82491 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:00.239913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f509-d825-4540-bea3-5cf13ec82491 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:00.239913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f509-d825-4540-bea3-5cf13ec82491 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:00.241542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 125/235] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:00.246529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:00.246944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:00.247128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:00.268712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:00.269899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:00.290960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea93c1-1948-4ec9-a432-8805e872bd7e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:00.291249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea93c1-1948-4ec9-a432-8805e872bd7e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:00.291506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea93c1-1948-4ec9-a432-8805e872bd7e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:00.293018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 126/236] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:00.298210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:00.298489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:00.298758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:00.331432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:00.333313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:00.354581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:00.397338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 111/237] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:00.448600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 127/238] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:00.715631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:00.757415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:00.776974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:00.832613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d96d82-b424-4ec5-adb0-b86299d7f320 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:00.868713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:00.869345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:00.869622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:00.869894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:00.871012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:00.872565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:00.874568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 112/239] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:00.881898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:00.987646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9543b2-46b3-408e-a72f-8f91d8f13891 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:01.063239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:01.066917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8645f5-67f5-4d61-97bc-3aaf9c20f7c0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:01.072586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:01.072586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.072586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.072586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.101417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:01.103531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:01.104139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.104440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.104705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.105350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:01.106764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:01.107625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:01.109297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:01.110160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:01.110242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 128/240] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:01.119209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:01.125246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:01.226862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d6cde-750d-4376-9f4e-bbc352a26924 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:01.242959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 113/241] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:00 2022] POST /identity/v3/users => generated 293 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:01.272622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:01.273303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.273614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.273886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.274606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:01.274913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:01.276370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:01.287093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 129/242] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:01.292481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:01.426660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aa342-68e6-40aa-bfb0-bfdc6c944c0e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:01.466368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:01.467018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.467294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.467565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] PATH_INFO: `/v3/users/2eaf643484454bff8753fa71b885b36d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.483809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf643484454bff8753fa71b885b36d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:01.485418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f3117-5abb-4bb0-afa8-0a91c2767085 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:01.488051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 114/243] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:01 2022] GET /identity/v3/users/2eaf643484454bff8753fa71b885b36d => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:01.494086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:01.566045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:01.602919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad09830-6241-42da-a4ab-4069c23b45a8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:01.604729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949b1e5-4d48-4e09-a14e-7d1911e93796 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.605013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949b1e5-4d48-4e09-a14e-7d1911e93796 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.605511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949b1e5-4d48-4e09-a14e-7d1911e93796 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.608020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 115/244] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:01.615338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a47e-594d-4066-81a8-1a51481f34bb None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.615683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a47e-594d-4066-81a8-1a51481f34bb None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.615964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a47e-594d-4066-81a8-1a51481f34bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.638260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-dbf9a47e-594d-4066-81a8-1a51481f34bb None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:01.639601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-dbf9a47e-594d-4066-81a8-1a51481f34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:01.650970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:01.651584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.651868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.652139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.654779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:01.738445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 116/245] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:01.740091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ef3b8-e45c-452e-8b32-e175d92b55d8 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.740411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ef3b8-e45c-452e-8b32-e175d92b55d8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.740692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ef3b8-e45c-452e-8b32-e175d92b55d8 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.742772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 117/246] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:01.748911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:01.749212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:01.749558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:01.755999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:01.757917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:01.770486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:01.772015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:01.840492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:01.876216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 118/247] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 25 02:47:01.886232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 130/248] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:01.893994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:01.990728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:02.021207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11db6dd-b831-47d7-94e8-dbe3b9fde099 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.071727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.072413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.072693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.072969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.074250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:02.075877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:02.076801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.077874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 119/249] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:01 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:02.084523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.212110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a03813-91ae-4e8f-a9f2-3c9650c1da5e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.249597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.250384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.250650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.250921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.262580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:02.268837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:02.269964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:02.270754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 131/250] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:02.276456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.293361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494cf5-3fd7-4d7c-b6ee-2c92c237c5d9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.328183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:02.350220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.351222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.351813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.352317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.353283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=2eaf643484454bff8753fa71b885b36d) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:02.353800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:02.355477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5299d2-ee76-4000-9a44-ecd7421c9831 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:02.366820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 120/251] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/role_assignments?user.id=2eaf643484454bff8753fa71b885b36d&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:02.368413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80835835-b538-4a2b-86f6-323fca2880f6 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.368907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80835835-b538-4a2b-86f6-323fca2880f6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.369387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80835835-b538-4a2b-86f6-323fca2880f6 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.371283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 121/252] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:02.372339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80835835-b538-4a2b-86f6-323fca2880f6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.390621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:02.434230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409bb-6f8b-4ac1-b55c-be32394bdbc4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.488953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.489579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.490068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.490470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.491293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:02.491596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:02.493142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876fd0c-f506-462a-8f0c-9bb07031b424 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:02.504857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 132/253] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:02.506140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.506419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.506691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.521885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80835835-b538-4a2b-86f6-323fca2880f6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.527536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:02.528834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:02.562393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.563441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.563957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.564482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.577319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:02.584852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:02.587196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:02.588000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:02.589432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 122/254] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:02.590517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.628619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 133/255] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:02.629568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.665531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:02.677926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:02.728067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea5cda-c6da-4a77-a0fd-90b1b4004de5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.807581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.808066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.808370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.808652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] PATH_INFO: `/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.809867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=b81bf7c64c7d4a5a84b6d3e0a9dd501b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:02.811268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:02.812953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 123/256] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:02.818970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:02.839917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585da5a4-1689-47d0-806b-65be9c203f3b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:02.883445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:02.884059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:02.884409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:02.884687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:02.885352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:02.885652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:02.887029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:02.898328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 134/257] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:02.905262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.029407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.040179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314f0c-02c6-42d8-8864-820e6aba45dc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.046599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c1414-8fc2-49da-be74-049332781ed3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.077934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.078484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.078867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.079150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.080295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.081940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.083984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 135/258] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:03.090151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.145410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.146117 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.146402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.146669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.152852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.154337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.156283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 124/259] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:02 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:03.157059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.162591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.197558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5ac50-a362-4d16-bbbd-fc259ef9fde5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.233300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.233967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.234245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.234516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.260853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.307273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.308797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.413507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 136/260] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 02:47:03.414167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.454858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9032d-07d5-4139-a18a-f5d5b5d4efc0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.465707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.517295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1f225-0b51-4d9f-aaa2-29de375d79f0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.542208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.542869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.543156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.543436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.553149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.553818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.554094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.554364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.561035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, user_id=b81bf7c64c7d4a5a84b6d3e0a9dd501b, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.562469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.568735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:03.575937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:03.578285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 125/261] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:03.580368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.648290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 137/262] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:03 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:03.649138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.659451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.736195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1799-ef0e-43a1-b994-a6301a406fdd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.774601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.775315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.775596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.775866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.777050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.778516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.780524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 126/263] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:03.786895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:03.854233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689249c-be51-4be9-a0c8-1b08cd7130cf None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:03.874239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:03.920282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:03.925846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:03.935671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 138/264] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:03.941209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:04.008257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ba856-45cf-4401-afd2-925afc45cf21 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:04.085349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7887c6d-7b40-477e-90cc-76233492204b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:04.090189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:04.104047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:04.104695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.105029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.105310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.106052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=1af36a254b9a41b4b9a1ac21a01adfe7) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:04.106363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:04.107901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:04.121509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 127/265] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/role_assignments?user.id=1af36a254b9a41b4b9a1ac21a01adfe7&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:04.141029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:04.141716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.142002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.142283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] PATH_INFO: `/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.144395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] RBAC: Authorizing `identity:get_user(user_id=1af36a254b9a41b4b9a1ac21a01adfe7)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:04.146063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554c82-29e6-4fa1-a13c-12c561853f38 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:04.147808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 139/266] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:03 2022] GET /identity/v3/users/1af36a254b9a41b4b9a1ac21a01adfe7 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:04.153915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:04.328291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8569867-99e7-4f45-afd2-746e3ae04b54 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:04.373194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:04.373928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.374221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.374498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.435472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4288-ed00-4499-bd10-49202fce1f71 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.436505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4288-ed00-4499-bd10-49202fce1f71 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.437006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d4288-ed00-4499-bd10-49202fce1f71 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.441061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 140/267] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:04.453638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.454255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.454755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.458036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:04.472826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:04.474344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791c21-f43f-44fc-b8f8-fe1d4dae5810 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:04.493795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:04.495517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:04.539637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:04.587045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 141/268] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:04.615229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 128/269] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:04.620942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:04.667144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6e33-2093-4bd6-ab29-66d960a2fb44 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.667469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6e33-2093-4bd6-ab29-66d960a2fb44 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.667725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6e33-2093-4bd6-ab29-66d960a2fb44 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.669110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 129/270] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:04.675123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.675468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.675721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.702269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:04.705013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:04.811878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 130/271] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:04.818765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75865e-3665-4d34-8e61-1e30ae46789a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:04.857238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:04.858123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:04.858551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:04.858969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:04.860288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:04.862047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67945408-53d9-4958-9ab7-c58a2044cf01 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:04.864230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 142/272] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:04 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:04.870399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:04.897551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:04.976703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447810-6d08-4169-8963-7c76b183b87d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.013147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.013768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.014048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.014321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.015059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=1af36a254b9a41b4b9a1ac21a01adfe7) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:05.015370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.016870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.023422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 131/273] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:04 2022] GET /identity/v3/role_assignments?user.id=1af36a254b9a41b4b9a1ac21a01adfe7&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:05.051366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8e5b-89a3-4d8b-b85b-191746cfca8a None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.051706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8e5b-89a3-4d8b-b85b-191746cfca8a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.051973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8e5b-89a3-4d8b-b85b-191746cfca8a None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.053382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 143/274] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:05.058510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.059624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.059912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.060208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.099368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:05.100920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:05.147171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.164063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.177804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb12de-b79a-4f49-a3ce-98a509c4ac89 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.221847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.224104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.224941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.225216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.227563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 144/275] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:05.229473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.240367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:05.245380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:05.246286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:05.247077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 132/276] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:05.253313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.361244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f6759-b2f8-4501-9d0d-f2f85488d65e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.380357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4001d-c5b4-43bd-bcbd-bef1587f3b95 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.403889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.404485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.404740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.405118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.408854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.410392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.412181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 145/277] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:05.419206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.430507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.431118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.431402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.431672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.432369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:05.432677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.434509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.445762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 133/278] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:05.451660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.469557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.473491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.559025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf4d02-041a-4abc-b8c2-8283966d5601 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.575696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dcfec-a299-46f6-8499-f7952d9fb9e2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.598513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.599120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.599396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.599769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.600416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.601868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.614136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.614872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.615157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.615433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] PATH_INFO: `/v3/users/e442f393850c4b3f9f1ea1c8786a13a1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.629919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] RBAC: Authorizing `identity:get_user(user_id=e442f393850c4b3f9f1ea1c8786a13a1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.631435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.633105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 134/279] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/users/e442f393850c4b3f9f1ea1c8786a13a1 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:05.637060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.702718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.728473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 146/280] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:05 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:47:05.729656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.755101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647778-eed8-41ef-ae3c-a056cceb1079 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.799573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.800541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.800929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.801293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.813898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:05.820234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:05.821215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:05.822029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:05.826617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 135/281] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:05.827714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:05.868774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:05.881572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529873-d2eb-41bd-8e5f-589354475130 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:05.926382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:05.928100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:05.928571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:05.929038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:05.930508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, user.id=e442f393850c4b3f9f1ea1c8786a13a1, scope.system=all) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:05.930993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:05.933636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:05.943171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 147/282] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/role_assignments?user.id=e442f393850c4b3f9f1ea1c8786a13a1&scope.system=all&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:05.962409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0ef86-97cc-4df3-a38a-581cdb9f7767 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:06.001580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:06.002236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.003207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.003881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.004434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:06.004747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:06.006572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:06.008464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:06.017354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 136/283] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:06.024027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:06.145962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3461a5-6655-45b3-be83-374a9c041310 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:06.182392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:06.183092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.183382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.183658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] PATH_INFO: `/v3/users/2eaf643484454bff8753fa71b885b36d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.187794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf643484454bff8753fa71b885b36d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:06.187927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4ad93-e616-4ae6-a2bf-94e1ad603168 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:06.190385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 148/284] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:06 2022] GET /identity/v3/users/2eaf643484454bff8753fa71b885b36d => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:06.196190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:06.343806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573a7d-98a4-4df7-817e-54db4a07d943 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:06.351841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de234ae4-d78f-45ba-be07-25f822f6edb2 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.352134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de234ae4-d78f-45ba-be07-25f822f6edb2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.352419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de234ae4-d78f-45ba-be07-25f822f6edb2 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.353867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 149/285] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:06.361445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.362257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.362532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.382482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:06.383120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.383401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.383675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.388905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:06.390386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:06.392054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 137/286] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:06 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:06.397900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:06.398038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:06.399112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:06.427846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:06.472873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:06.521503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b688cc-8abf-4623-9725-cde87d51dfc4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:06.555429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 150/287] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:06.572883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:06.573566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:06.573850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:06.574125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/users/2eaf643484454bff8753fa71b885b36d/roles/d781c665e4d34e4e8014866ef91bdd52` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:06.577785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, user_id=2eaf643484454bff8753fa71b885b36d, role_id=d781c665e4d34e4e8014866ef91bdd52)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:06.579240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:06.669826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 138/288] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:06 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/users/2eaf643484454bff8753fa71b885b36d/roles/d781c665e4d34e4e8014866ef91bdd52 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:47:06.838703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:06.961651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9521ce-f657-4948-8ace-dfc0e2103a42 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:07.000602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:07.001247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:07.001528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:07.001804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:07.015134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:07.020789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:07.021667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:07.022440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 151/289] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:07.027947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:07.173774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bcc13-97b2-4f19-b519-a9d10db60b75 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:07.192256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:07.218816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:07.219440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:07.219724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:07.219991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:07.220708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:07.221095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:07.222985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:07.236342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 139/290] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:07.241750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:07.275238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:07.405005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3b922-0813-46ac-93aa-fd6762586e5d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:07.457833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:07.458398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:07.458682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:07.458958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] PATH_INFO: `/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:07.460104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] RBAC: Authorizing `identity:get_user(user_id=b81bf7c64c7d4a5a84b6d3e0a9dd501b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:07.461576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:07.463455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 152/291] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:07 2022] GET /identity/v3/users/b81bf7c64c7d4a5a84b6d3e0a9dd501b => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:07.469138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:07.627006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf29ee-3020-4db9-9fd1-5857103c4b0b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:07.673009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:07.673628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:07.673913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:07.674269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:07.747679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:07.773182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:07.774687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:07.927735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 140/292] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:07.933289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:08.062290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:08.089274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd63ba-4f97-46dc-9da3-5024cd944e95 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:08.135184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:08.135829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.136111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.136413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.137566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:08.139008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8a368-a9f7-4836-941c-f9900c848d18 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:08.141643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 153/293] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:07 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:08.146698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:08.256605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:08.265996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974973-6c0d-4db1-a8da-1ba39e129a42 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.266286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974973-6c0d-4db1-a8da-1ba39e129a42 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.266698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2974973-6c0d-4db1-a8da-1ba39e129a42 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.268046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 154/294] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:08.273228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5980-13ed-49a3-8a4d-a4904231db8b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.274767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5980-13ed-49a3-8a4d-a4904231db8b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.274767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5980-13ed-49a3-8a4d-a4904231db8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.295980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14579fd0-9b88-4594-b0de-fc167d7ff918 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:08.303282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-865f5980-13ed-49a3-8a4d-a4904231db8b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:08.304521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-865f5980-13ed-49a3-8a4d-a4904231db8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:08.333407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:08.334374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.334849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.335313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.336372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=b81bf7c64c7d4a5a84b6d3e0a9dd501b) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:08.336946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:08.338661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74761b4a-adbd-471f-bafc-48720a8b6db5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:08.345043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 141/295] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:08 2022] GET /identity/v3/role_assignments?user.id=b81bf7c64c7d4a5a84b6d3e0a9dd501b&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:08.347130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e124a-a5ed-4466-9da0-e8d673592238 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.347650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e124a-a5ed-4466-9da0-e8d673592238 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.348176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e124a-a5ed-4466-9da0-e8d673592238 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.350017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 142/296] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:08.360782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.360782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.360782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.379760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:08.381952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:08.445687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 155/297] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:08.447141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e87d-aa5e-4896-a1b9-76e56ab3c5eb None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.447522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e87d-aa5e-4896-a1b9-76e56ab3c5eb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.447891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e87d-aa5e-4896-a1b9-76e56ab3c5eb None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.449306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 156/298] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:08.465064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 143/299] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:08.471493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.472072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.472483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.504511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:08.505717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:08.589246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 157/300] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:08.749230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:08.771836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:08.788451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:08.851804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:08.883456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d0c99-7b2a-4c11-9296-3483f6ae387d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:08.903220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48d60-3f4a-41a6-99f9-0479b120991a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:08.922426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:08.923070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.923353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.923634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.925348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:08.932814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:08.932814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 144/301] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:08.932814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:08.949136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:08.949772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:08.949997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:08.950391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:08.961527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:08.966799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:08.967702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 158/302] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:08.968866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.038503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.051687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e6a2-7e5e-4737-ba14-7e0104f09d9f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.072565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2acef-0cdf-4f3f-a7c9-7b8baf46d6ff None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.089220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.090107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.090560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.090973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.102380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:09.107774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:09.108829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:09.109825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:09.112365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 145/303] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:09.112487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.113644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.116336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.117887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.118002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.118227 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] PATH_INFO: `/v3/groups` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.118871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.120346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.213195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 159/304] 10.222.0.75 () {60 vars in 1134 bytes} [Thu Aug 25 02:47:08 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:09.214361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.215027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b6-7d56-453b-bbeb-58f17782ec2a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.276118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.277263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.277739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.278253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.279538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:09.280364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.282170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.293740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 146/305] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:09.300554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.368222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50c289-dbd0-4ee6-b94b-084bf93abdbe None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.387458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.409292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.415119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.416005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.416507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.416975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.417904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:09.418431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.419870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d53ce-3c23-45e7-b52a-21196e4dc42b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.432186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 160/306] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:09.432348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0accc-844d-4bec-ab47-d9f98ca560bf None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.432430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9675b-0d0a-44dc-b4f7-2c9a0a05d156 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.432430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9675b-0d0a-44dc-b4f7-2c9a0a05d156 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.432430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9675b-0d0a-44dc-b4f7-2c9a0a05d156 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.434661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 161/307] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:09.437006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9675b-0d0a-44dc-b4f7-2c9a0a05d156 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.469221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.469852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.470128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.470398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] PATH_INFO: `/v3/users/e442f393850c4b3f9f1ea1c8786a13a1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.472298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] RBAC: Authorizing `identity:get_user(user_id=e442f393850c4b3f9f1ea1c8786a13a1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.473789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9e573-fde1-4ee0-a701-db2923572a82 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.475371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 147/308] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/users/e442f393850c4b3f9f1ea1c8786a13a1 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:09.479971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.479971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.479971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.495566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:09.496795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:09.556710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9675b-0d0a-44dc-b4f7-2c9a0a05d156 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.567616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 148/309] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:09.568480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.601106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.603923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.603923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.603923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] PATH_INFO: `/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.619556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] RBAC: Authorizing `identity:get_user(user_id=f2c7d78e2f9b4481a55cd6e66c4a3ce1)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.621394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.629094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 162/310] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:09.630641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:09.632889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.672434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de0be-4468-46e6-b0c8-878128fae398 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.686650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:09.707790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.708476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.708751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.709020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] PATH_INFO: `/v3/system/users/e442f393850c4b3f9f1ea1c8786a13a1/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.710436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e442f393850c4b3f9f1ea1c8786a13a1, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.711818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.743920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50480-45d4-496b-a4ca-943e74510153 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:09.780555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:09.781158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:09.781433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:09.781699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:09.782411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, user.id=f2c7d78e2f9b4481a55cd6e66c4a3ce1, scope.system=all) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:09.782864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:09.784129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:09.789455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 163/311] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/role_assignments?user.id=f2c7d78e2f9b4481a55cd6e66c4a3ce1&scope.system=all&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:09.819423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 149/312] 10.222.0.75 () {58 vars in 1334 bytes} [Thu Aug 25 02:47:09 2022] PUT /identity/v3/system/users/e442f393850c4b3f9f1ea1c8786a13a1/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:09.861254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.002398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4235-076d-4a6b-b5bd-e1837d434136 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:10.017171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:10.041603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:10.042663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:10.043155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:10.043625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:10.056353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:10.061462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:10.062327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:10.063110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:10.066303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 164/313] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:10.071857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.183382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529d1cf-387d-4b3e-a900-bb3348794793 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:10.213050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:10.223346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:10.229853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:10.238147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 150/314] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:10.243650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.363075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd437-e40b-428f-a077-ddc76971a631 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:10.398880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:10.399516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:10.399789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:10.400057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] PATH_INFO: `/v3/users/2eaf643484454bff8753fa71b885b36d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:10.401209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2eaf643484454bff8753fa71b885b36d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:10.402650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:10.404552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 165/315] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:10 2022] GET /identity/v3/users/2eaf643484454bff8753fa71b885b36d => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:10.410746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.516937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95941c9-c983-475f-a1c2-6fd0bb8b1f60 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:10.552529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:10.553147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:10.553431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:10.553703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:10.613508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:10.624135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:10.625668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:10.730573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 151/316] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:10.735773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.780680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:10.849068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e86ca-218e-4fe9-8a94-8261981189c6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:10.886516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:10.887163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:10.887438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:10.887710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:10.888906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:10.890359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228eb0e4-f317-4f3f-a389-1c3ee5b0ab9a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:10.891988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 166/317] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:10 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:10.898881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:10.991683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:11.005379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500783f-4679-4850-b964-50e95c160d92 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:11.045853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:11.046390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.046966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.047247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.047972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, scope.project.id=d5805663da164311887839f7a5cfa666, user.id=2eaf643484454bff8753fa71b885b36d) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:11.048316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:11.049799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d76323-d121-4fa6-8c89-f6e4725ef754 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:11.055734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 152/318] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:10 2022] GET /identity/v3/role_assignments?user.id=2eaf643484454bff8753fa71b885b36d&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:11.247688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db6f2-fd00-4eb1-8888-62e998c73693 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.248201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db6f2-fd00-4eb1-8888-62e998c73693 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.248309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db6f2-fd00-4eb1-8888-62e998c73693 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.250045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 167/319] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:11.256096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314635-a78b-47b4-abff-0434c06df708 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.256448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314635-a78b-47b4-abff-0434c06df708 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.256734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80314635-a78b-47b4-abff-0434c06df708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.276445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-80314635-a78b-47b4-abff-0434c06df708 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:11.277644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-80314635-a78b-47b4-abff-0434c06df708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:11.348161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 153/320] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:11.618826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722db6f2-fd00-4eb1-8888-62e998c73693 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:11.722457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722db6f2-fd00-4eb1-8888-62e998c73693 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:11.731962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:11.742817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6719-ef18-46d8-8f75-aa54101b3579 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.743110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6719-ef18-46d8-8f75-aa54101b3579 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.746053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6719-ef18-46d8-8f75-aa54101b3579 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.746053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 154/321] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:11.749660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b923e-5029-4335-9ab2-cab63fd76f3d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.749944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b923e-5029-4335-9ab2-cab63fd76f3d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.750217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b923e-5029-4335-9ab2-cab63fd76f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.758554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:11.759206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.759483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.759754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.761250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:11.763072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:11.769057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-043b923e-5029-4335-9ab2-cab63fd76f3d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:11.769624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 168/322] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:11.770389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-043b923e-5029-4335-9ab2-cab63fd76f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:11.770487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:11.838143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 155/323] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:11.841699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1555-5469-47bc-bec6-a43f6414ec5f None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.842593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1555-5469-47bc-bec6-a43f6414ec5f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.842871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1555-5469-47bc-bec6-a43f6414ec5f None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.844192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 156/324] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:11.849345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.849627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.849899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.867524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:11.868725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:11.876013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:11.879143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed0e66-c689-466a-8e56-21bab601cd42 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:11.922625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:11.923245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:11.923528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:11.923850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] PATH_INFO: `/v3/groups` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:11.925198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:11.929035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:11.944105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 157/325] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:12.040189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 169/326] 10.222.0.75 () {60 vars in 1134 bytes} [Thu Aug 25 02:47:11 2022] POST /identity/v3/groups => generated 226 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:12.106714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.213571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ae7f8-1103-4f7b-a415-b9991c81f067 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.221966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.254801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.255766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.256051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.256350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.266780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.267235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:12.273026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:12.273898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:12.274697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 158/327] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:12.280015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.285334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.352116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b2933-a73b-45d4-beca-ff66949e1966 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.397474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.398518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.399180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.399465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.400792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ca4c3-ac0a-4d5a-8dcd-2e8382a65ac9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.413196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:12.419907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:12.420892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:12.421985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 170/328] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:12.428597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.449741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.450356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.450751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.451023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.451818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:12.452122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:12.453564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:12.465706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 159/329] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:12.473872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.548866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.558681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.567009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c4021-4e39-49a9-bd0c-50d4a5971d36 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.598972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fc6e-f72e-4dfe-9341-7c059ea4fd41 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.608866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.609841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.610128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.610405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.611956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:12.613484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:12.619887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:12.628248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 171/330] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:12.636496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.643990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.644668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.644946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.645216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] PATH_INFO: `/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.646524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] RBAC: Authorizing `identity:get_user(user_id=f2c7d78e2f9b4481a55cd6e66c4a3ce1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:12.647966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:12.649866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 160/331] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:12.655941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.790281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283421f-0d60-435f-b6a2-3db447ac75cc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.801175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa49d1-a332-45d8-a449-4165c2f864e9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:12.838283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.839094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.839786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.840063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.840359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] PATH_INFO: `/v3/users/e442f393850c4b3f9f1ea1c8786a13a1` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.841531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] RBAC: Authorizing `identity:get_user(user_id=e442f393850c4b3f9f1ea1c8786a13a1)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:12.842991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:12.844768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 172/332] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/users/e442f393850c4b3f9f1ea1c8786a13a1 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:12.850623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:12.851283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:12.854708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:12.854708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] PATH_INFO: `/v3/system/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1/roles/d781c665e4d34e4e8014866ef91bdd52` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:12.854972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:12.855064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f2c7d78e2f9b4481a55cd6e66c4a3ce1, role_id=d781c665e4d34e4e8014866ef91bdd52)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:12.855064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457191f1-1dd4-4a8b-a3a3-fc0f70f4bf1d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:12.887021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:12.921123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 161/333] 10.222.0.75 () {58 vars in 1334 bytes} [Thu Aug 25 02:47:12 2022] PUT /identity/v3/system/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1/roles/d781c665e4d34e4e8014866ef91bdd52 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:12.987715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea2fa8-2350-4623-aac3-bc6573b8dce9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:13.027861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:13.028524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:13.028799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:13.029069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:13.029818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] RBAC: Adding query filter params (role.id=5baef496d0274b11922ddd2613e3ff1a, user.id=e442f393850c4b3f9f1ea1c8786a13a1, scope.system=all) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:13.030121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:13.031615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a995e-4bb8-46d9-9cf7-7d3419bb999f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:13.044884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 173/334] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Aug 25 02:47:12 2022] GET /identity/v3/role_assignments?user.id=e442f393850c4b3f9f1ea1c8786a13a1&scope.system=all&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:14.307160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa040-8f23-4908-88bf-e5c88ffc9273 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:14.308029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa040-8f23-4908-88bf-e5c88ffc9273 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:14.308503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa040-8f23-4908-88bf-e5c88ffc9273 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:14.310021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 162/335] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:14.316972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3b86-fe20-4032-926a-ba9a592b47ca None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:14.317857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3b86-fe20-4032-926a-ba9a592b47ca None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:14.318301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3b86-fe20-4032-926a-ba9a592b47ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:14.341113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-16ff3b86-fe20-4032-926a-ba9a592b47ca None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:14.342600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-16ff3b86-fe20-4032-926a-ba9a592b47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:14.428632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 174/336] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:14.489365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a96c8b-48e1-4453-8a51-621b75af7d0e None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:14.489930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a96c8b-48e1-4453-8a51-621b75af7d0e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:14.490392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a96c8b-48e1-4453-8a51-621b75af7d0e None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:14.491871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 163/337] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:14.497545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:14.497825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:14.498096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:14.522470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:14.523658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:14.534175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:14.609930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 175/338] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:14.743166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a96c8b-48e1-4453-8a51-621b75af7d0e None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:14.868915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a96c8b-48e1-4453-8a51-621b75af7d0e None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:14.911292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:14.911922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:14.912219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:14.912492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:14.940362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:14.945737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:14.946630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:14.947419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 164/339] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:14.957260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:14.999045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:15.083462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f1a75-450f-46db-abd7-20f8e64b0a2c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.088185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.141910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:15.142699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.142979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.143258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.143934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:15.144258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:15.145632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:15.153522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affd5-3d40-48b6-b6e7-5d7475609cc9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.154431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 176/340] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:15.160611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:15.172909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.228617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:15.229645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.230135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.230612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.263434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:15.269725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:15.270620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:15.271441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:15.280616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 165/341] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:15.289358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:15.303522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f347ef-47eb-40da-9c7d-051d6f9a53dc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.349490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:15.350113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.350386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.350654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.475659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef42ac-4517-482c-b7c4-722b3eff99f9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.495714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.521752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:15.521963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:15.522375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.522654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.522924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.523244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:15.523589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:15.523892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:15.525280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:15.538322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 166/342] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:15.542081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.544029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:15.695153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef1fb7-8bef-45ca-b7d1-1d2135ce824c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.704855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 177/343] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 02:47:15.722106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:15.740327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:15.740855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.741133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.741399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.806523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.830042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:15.832028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ffb95-5420-47e8-a767-b334e6223103 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:15.942077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aee8-88b9-4965-a9f6-3205bdccc0c6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:15.956896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 167/344] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:15.958180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784dd36-5500-44b7-b4b1-f0c1627a46a6 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.958461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784dd36-5500-44b7-b4b1-f0c1627a46a6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.958731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784dd36-5500-44b7-b4b1-f0c1627a46a6 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.959972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 168/345] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:15.963658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:15.969519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9507827-72f3-42cb-b357-46b700534289 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:15.969804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9507827-72f3-42cb-b357-46b700534289 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:15.970079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9507827-72f3-42cb-b357-46b700534289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:15.996741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c9507827-72f3-42cb-b357-46b700534289 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:15.998529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c9507827-72f3-42cb-b357-46b700534289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:16.001411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.002690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.002888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.003142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.004444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.005915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.008909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 178/346] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:15 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:16.009710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.136734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 169/347] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:16.137811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9507827-72f3-42cb-b357-46b700534289 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.162491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:16.199512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3954b-1485-4ac0-9649-2d1789057aae None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.241879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.242530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.242805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.243075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.244416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.245858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bff9e1-dee8-4f12-9234-e724fda84404 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.248765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 179/348] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:16.251441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90d526-dad2-4fb0-a9f8-75030398ab24 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.251723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90d526-dad2-4fb0-a9f8-75030398ab24 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.257295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90d526-dad2-4fb0-a9f8-75030398ab24 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.261985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 180/349] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:16.262625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90d526-dad2-4fb0-a9f8-75030398ab24 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.346564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:16.359190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9507827-72f3-42cb-b357-46b700534289 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.414969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90d526-dad2-4fb0-a9f8-75030398ab24 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.441052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.441755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.442040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.442311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.451999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.453497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea78f-6af9-42b4-958e-a8ec5e0c7a52 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.454533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.455389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.455619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 170/350] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:16.455787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.456046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.461159 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.462602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.464371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 181/351] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:16.465059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.465701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.465995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.466328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.487806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:16.489109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:16.577138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 171/352] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:16.578454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.582832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:16.629156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265588-b4ef-4d4c-82ef-ae77a5711621 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.643996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:16.672469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.673359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.673939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.674395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.675267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:16.675721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.678058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0c70a-cf98-4965-9dd8-a8d8ae0d87f1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.686195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 182/353] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:16.687563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0680b7-f1d5-4e98-b345-629b7aca83b7 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.688299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0680b7-f1d5-4e98-b345-629b7aca83b7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.688717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0680b7-f1d5-4e98-b345-629b7aca83b7 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.691930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45ef7-7037-4e26-be5c-85e28dabc8dd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.692363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 183/354] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:16.693624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0680b7-f1d5-4e98-b345-629b7aca83b7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.763188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.763871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.764183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.764453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.765209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:16.765522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:16.767017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356f06-12b2-4354-9426-bd6b3e2255ab None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:16.783984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 172/355] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:16.787585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.787882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.788203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.824653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:16.825900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:16.910136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0680b7-f1d5-4e98-b345-629b7aca83b7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:16.972736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:16.975868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:16.976689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:16.977087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:16.977458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:16.983597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 173/356] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:16.985043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:16.990773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:16.996060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:16.997093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:16.997907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 184/357] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:17.005088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.045012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.110153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ae6bb-fa44-47bb-bee8-1d196773ea49 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.116158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff4c6-5f35-4cb9-8c2d-eefaada1141b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.151325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.152476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.153007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.153644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.156505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.157290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.157654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.158354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.159097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:17.159606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.161054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c1e3d-f59e-4abc-97dc-0fa9295cfc06 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.168577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:17.170649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 185/358] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:17.172427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dbbfb-cfd1-48d1-ab5d-42ed046edc36 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.172848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dbbfb-cfd1-48d1-ab5d-42ed046edc36 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.173212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dbbfb-cfd1-48d1-ab5d-42ed046edc36 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.175149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 186/359] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c20b96d9-906d-44d5-8610-2af76905ade2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:17.175488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:17.176540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 25 02:47:17.177327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:17.178016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dbbfb-cfd1-48d1-ab5d-42ed046edc36 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.179962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 174/360] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:17.181513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.181974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.182419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.208708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:17.210214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:17.267755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.288503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 175/361] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:17.289709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.304612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.311589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dbbfb-cfd1-48d1-ab5d-42ed046edc36 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.348007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.350332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.350696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.351110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.411177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3b746-90d2-44d6-bae9-9d7027993c0a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.421824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.423363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.452769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.453678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.454134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.454608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.456098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:17.456616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.458197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.468061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 176/362] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:17.478356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.552512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.577693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.583300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 187/363] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:17.588244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.638100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53cf59-5f80-4359-9c68-815cd44a09f8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.696833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.697401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.697682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.697958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] PATH_INFO: `/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.699870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] RBAC: Authorizing `identity:get_user(user_id=f2c7d78e2f9b4481a55cd6e66c4a3ce1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.702255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.706547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 177/364] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/users/f2c7d78e2f9b4481a55cd6e66c4a3ce1 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:17.707264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.736747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe140-fe76-433b-ad6d-83ca53996ea2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.779835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.780541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.780822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.782616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.792743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:17.800196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:17.801135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 188/365] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:17.804782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.851634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531e55-4c1e-4da6-a98d-bd4dfcb51f87 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.870863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.872535 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:17.895072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.896035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.896538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.897110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.898473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.900128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.902004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 178/366] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:17.903020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:17.924752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24f3-fd65-4763-88ce-2f66cbd50668 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:17.961708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:17.962240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:17.962521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:17.963413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:17.963531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] RBAC: Adding query filter params (role.id=d781c665e4d34e4e8014866ef91bdd52, user.id=f2c7d78e2f9b4481a55cd6e66c4a3ce1, scope.system=all) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:17.963825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:17.966364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:17.981240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 189/367] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/role_assignments?user.id=f2c7d78e2f9b4481a55cd6e66c4a3ce1&scope.system=all&role.id=d781c665e4d34e4e8014866ef91bdd52 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:17.982182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.024905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a756f-271f-4656-8789-93b8f58f01cf None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.065306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.066600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.067124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.067595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.082995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:18.090371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:18.091258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:18.092040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:18.093540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 179/368] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:18.093540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.115741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3aaa1-52c4-45c0-a9d3-632cd4559c8d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.135740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.160821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.161487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.161771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.162047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.162721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:18.163030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.164460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.164965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.179455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 190/369] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:18.182564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.234505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa8c0-4957-4d01-9a30-fce13d1d25a9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.276267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.276913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.277196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.277474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.278771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.280394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.282107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 180/370] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:18.282902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.327864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052872-db43-471d-a3ad-8801ec1149ba None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.373220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.374963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.375430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.375881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.376822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:18.378008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.379789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.391400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 191/371] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:18.393394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.416077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fb2d3-a4ff-45ef-bb61-fab66edff6b8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.435407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.463256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.478343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.478968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.479247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.479942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.545717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a129acb-247c-4b2b-9ce3-b4c5a81a95bf None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.573827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.575326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.592795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.593073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.593817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.594810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=d5805663da164311887839f7a5cfa666) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:18.595122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.597307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.604708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 192/372] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:18.605750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.731923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45cf33-0c80-4dba-8c7e-337858fd2569 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:18.734350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.776284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:18.777356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:18.777638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:18.778050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:18.783787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 181/373] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:18.789726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:18.821039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:18.864766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:18.866701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:18.959929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7f993-5d60-4e32-b99e-d2a96419a965 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.000485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.001357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 193/374] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:19.001646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.002144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.002444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.003700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.006576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.007094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 182/375] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:18 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:19.007891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:19.010099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:19.016169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:19.162032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:19.199323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b70-3631-413a-a467-9a1da6530cc0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.217304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7010e-d1f5-4870-9dee-dcd1a5e0f932 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.273798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.274460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.274744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.275020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.276615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.277504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.277987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.278461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] PATH_INFO: `/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.279832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] RBAC: Authorizing `identity:get_project(project_id=eabb40ce2ddb40b4bea318792fc09f8e)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.280891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.281758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.282353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.284633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 194/376] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:19 2022] GET /identity/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:19.286572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 183/377] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:19 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:19.293189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:19.293322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:19.379931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:19.415963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42957-17ae-4ff4-8ca7-46c244c832f2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.456712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.457698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.458171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.458632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] PATH_INFO: `/v3/groups/00c9fbf78e6144c797ac625539e7ec5d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.466559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] RBAC: Authorizing `identity:get_group(group_id=00c9fbf78e6144c797ac625539e7ec5d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.468827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.471654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 195/378] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:19 2022] GET /identity/v3/groups/00c9fbf78e6144c797ac625539e7ec5d => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:19.478679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:19.527720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06fd3-2931-4111-943b-db23291fd926 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.583608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.584349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.585007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.585007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.585852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=d5805663da164311887839f7a5cfa666) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:19.586164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.587804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf435-2337-41a8-be6c-a12e5bf641c1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.595205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 184/379] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:19 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 260 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:19.596758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf48e2-b937-46a9-a464-8b83813cd0e2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.597066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf48e2-b937-46a9-a464-8b83813cd0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.597529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf48e2-b937-46a9-a464-8b83813cd0e2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.603436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 185/380] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:19.603436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f65bd-a6db-4e25-9243-d40f0efc565e None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.603436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f65bd-a6db-4e25-9243-d40f0efc565e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.603436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f65bd-a6db-4e25-9243-d40f0efc565e None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.605345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 186/381] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:19.606131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.606417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.606690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.630460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79978751-e9b4-4437-b45c-3e5cc21e19ff None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:19.632488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:19.634902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:19.648600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:19.682974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:19.683880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.684252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.684530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.685453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] RBAC: Adding query filter params (group.id=00c9fbf78e6144c797ac625539e7ec5d, role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=eabb40ce2ddb40b4bea318792fc09f8e) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:19.686172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:19.687936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c7d9f-c66e-4d55-ab9b-e1004906e65d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:19.690664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:19.698006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 196/382] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:19 2022] GET /identity/v3/role_assignments?group.id=00c9fbf78e6144c797ac625539e7ec5d&scope.project.id=eabb40ce2ddb40b4bea318792fc09f8e&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:19.704195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:19.705775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:19.705775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:19.739550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:19.745096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:19.750186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 187/383] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:19.828861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 197/384] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:20.086775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.206897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de4455-0c08-4c88-a4d3-526ebbce476d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.231023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.273712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.275407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.276089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.277323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.307893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:20.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:20.313877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:20.314715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 188/385] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:20.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.328047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:20.394874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfe58f-87cb-4c6c-aea8-d666fa27d303 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.452515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:20.468407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.469148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.469433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.469702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.493309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:20.503833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:20.504785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:20.505993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 198/386] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:20.511488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.523294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d8add-a31a-47ee-abe1-b73cc46e55b1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.602015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.602761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.602942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.603216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.603899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:20.604234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:20.605625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:20.626591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 189/387] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:20.630826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.690862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a16631-e733-4338-a468-886dbc354dc4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.733423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.734542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.734824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.735097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.735764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:20.736749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:20.738208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:20.747064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 199/388] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:20.752571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.782451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:20.816586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:20.830243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa821f-ae38-410d-b596-9f67502c9e3c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.873805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.874493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.874778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.875105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.876278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:20.878480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:20.882287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 190/389] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:20.882429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ffc-c524-439f-b029-504a12863fc1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:20.889173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:20.936605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:20.937994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:20.940518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:20.940518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:20.941378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:20.944178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:20.953138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 200/390] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:20.954811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:21.068725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063506e7-b7c6-425f-95c1-f5af62f3a4a4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:21.137209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:21.146681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:21.147334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.147613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.147887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.149081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:21.150540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c4152-3382-44f2-8dcd-045a8bc542f7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:21.155976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad1c71-53de-4708-9250-8db635b6e808 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:21.156089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 191/391] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:21.156089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4ba9-5421-4ea2-81d5-781bde35f21d None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.156089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4ba9-5421-4ea2-81d5-781bde35f21d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.156089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4ba9-5421-4ea2-81d5-781bde35f21d None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.156089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 192/392] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:21.162245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.162536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.162811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.172471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:21.189872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:21.191555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:21.191665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:21.192310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.192657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.192943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.194142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:21.195668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:21.197782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 201/393] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:20 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:21.198612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:21.311627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22267e1-c4bc-440c-a6a3-89bb40cf46bc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:21.323720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 193/394] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:21.324871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:21.348921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:21.349592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.349874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.350187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/groups/adaa3b34edb242588d15300aeec9da83/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.352443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, group_id=adaa3b34edb242588d15300aeec9da83, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:21.354068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:21.440634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 202/395] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Aug 25 02:47:21 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/groups/adaa3b34edb242588d15300aeec9da83/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:21.452889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbba02c-eafc-4a0f-b764-2e13009b0fa9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:21.488595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:21.500090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:21.500881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.501165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.501494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/groups/adaa3b34edb242588d15300aeec9da83/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.506149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] RBAC: Authorizing `identity:create_grant(project_id=c608b62797cb4b1aad77c875f6d5c5cb, group_id=adaa3b34edb242588d15300aeec9da83, role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:21.507783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:21.568123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 194/396] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Aug 25 02:47:21 2022] PUT /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb/groups/adaa3b34edb242588d15300aeec9da83/roles/ee78bca73e4446e2b38a9e195677f064 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:21.680176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:21.735530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:21.797729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040a6b5-fec0-4c61-9c98-ca3772d59c3c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:21.835175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:21.836172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:21.836655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:21.837125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:21.849384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:21.855007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:21.855971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:21.856801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:21.859423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 203/397] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:21.865408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:21.995365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754e56e-f776-480d-83df-20dbfa12657a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:22.054278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:22.055323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.055795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.056381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.057345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:22.057928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:22.059726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344741-5378-45d9-8065-ef14e9d73057 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:22.070571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 195/398] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:22.076397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:22.183114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f790bb-3e48-4641-a019-f487eec5e4a7 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.183801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f790bb-3e48-4641-a019-f487eec5e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.184293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f790bb-3e48-4641-a019-f487eec5e4a7 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.186140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 196/399] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:22.195597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920294-505c-4024-a7a0-336950aff9b3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.196307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920294-505c-4024-a7a0-336950aff9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.196425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a316-60d2-4933-b31d-c50cfc0ef14e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:22.196893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920294-505c-4024-a7a0-336950aff9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.205215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:22.220040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a0920294-505c-4024-a7a0-336950aff9b3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:22.224290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a0920294-505c-4024-a7a0-336950aff9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:22.256685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:22.257811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.259402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.259936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.262145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:22.266632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:22.269000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 204/400] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:22 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:22.275874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:22.322237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:22.362066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 197/401] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:22.363871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810178c-02cd-47dc-88d8-a699b1d72336 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.368501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810178c-02cd-47dc-88d8-a699b1d72336 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.369051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810178c-02cd-47dc-88d8-a699b1d72336 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.370929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 198/402] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:22.378227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.378799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.379266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.406296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:22.407501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:22.457664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbb9ca-8ead-4473-a91e-e37b4de12e99 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:22.498676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:22.499320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.499600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.499872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.501029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:22.503352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:22.505392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 205/403] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:22 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:22.511987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:22.554736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 199/404] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:22.562246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:22.634709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6aa8e-1824-4039-b751-250b21b46b69 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:22.680723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:22.681485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.681650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.681927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/groups/adaa3b34edb242588d15300aeec9da83/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.683826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, group_id=adaa3b34edb242588d15300aeec9da83, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:22.685298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:22.742850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:22.761120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 206/405] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Aug 25 02:47:22 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/groups/adaa3b34edb242588d15300aeec9da83/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:22.900471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea1d66-4f2b-41b2-a3af-99be4b6a6505 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:22.920867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:22.941632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:22.953924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:22.954585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:22.955409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:22.955682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:22.978750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:22.987320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:22.988411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:22.989210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:22.990410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 200/406] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:22.994561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.119188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856c7b3-3e6c-442a-911b-9070d9f557ce None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.120779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:23.128876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801169f-b9d2-4df7-8759-e186a6472bed None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.169607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.170762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.171611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.172197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.174698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:23.175403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.177571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.177830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.178478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.178760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.179036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.192629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 201/407] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:23.201174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.202094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:23.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:23.210411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:23.211581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 207/408] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:23.219127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.272769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:23.382596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e1ec2-f9fa-488e-82ab-32f531dd24b7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.419332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.420034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.420334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.420622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.421549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:23.421856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.424228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.433711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 208/409] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:23.441598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788217-1d72-4008-b4b8-d69dda7bd3ec None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.443972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.484579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:23.492842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.493487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.493765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.494039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.495182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.496666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.498408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 202/410] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:23 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:23.506441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.573033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf66968-2c82-4986-856c-0458348a8f1a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.617017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.617659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.617940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.618213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] PATH_INFO: `/v3/groups/00c9fbf78e6144c797ac625539e7ec5d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.620077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] RBAC: Authorizing `identity:get_group(group_id=00c9fbf78e6144c797ac625539e7ec5d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.622283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.628324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 209/411] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:23 2022] GET /identity/v3/groups/00c9fbf78e6144c797ac625539e7ec5d => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:23.634467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.753804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b56e0-6554-4259-ab1a-58b04958585f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.756405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:23.816976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.818000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.818483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.818943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.822119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.823784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.827867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 203/412] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:23 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:23.832716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:23.864525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:23.868818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63e078-dd01-4b69-b496-5dd1bb03222d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:23.932694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:23.933333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.933608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.933879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] PATH_INFO: `/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.935003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] RBAC: Authorizing `identity:get_project(project_id=eabb40ce2ddb40b4bea318792fc09f8e)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:23.936446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8d66-433d-41f6-8b11-fe8311bf462d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:23.938430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 210/413] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:23 2022] GET /identity/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:23.940524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae00b6e-731a-4f5e-a02a-53fa8bb57e3f None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:23.940524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae00b6e-731a-4f5e-a02a-53fa8bb57e3f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:23.940524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae00b6e-731a-4f5e-a02a-53fa8bb57e3f None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:23.944187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 211/414] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:23.946494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae00b6e-731a-4f5e-a02a-53fa8bb57e3f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:24.016357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faefe8a-420f-4981-b94e-a8b83ce8ce16 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:24.059426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae00b6e-731a-4f5e-a02a-53fa8bb57e3f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:24.065696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:24.066967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.067621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.068287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666/groups/adaa3b34edb242588d15300aeec9da83/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.070842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5805663da164311887839f7a5cfa666, group_id=adaa3b34edb242588d15300aeec9da83, role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:24.072770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ca970-3a29-41d2-a156-c200227b35d6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:24.099550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:24.100244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.100523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.100792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] PATH_INFO: `/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e/groups/00c9fbf78e6144c797ac625539e7ec5d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.102700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] RBAC: Authorizing `identity:create_grant(project_id=eabb40ce2ddb40b4bea318792fc09f8e, group_id=00c9fbf78e6144c797ac625539e7ec5d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:24.104126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0261d-2b6c-42e3-a3b4-0ea1e947bc4e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:24.145260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 204/415] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Aug 25 02:47:23 2022] PUT /identity/v3/projects/d5805663da164311887839f7a5cfa666/groups/adaa3b34edb242588d15300aeec9da83/roles/ee78bca73e4446e2b38a9e195677f064 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:47:24.147001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.147506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.147985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.168859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 212/416] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Aug 25 02:47:23 2022] PUT /identity/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e/groups/00c9fbf78e6144c797ac625539e7ec5d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:24.185450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:24.186966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:24.235795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:24.252518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ed1a-128f-4de4-94e9-50768b536a75 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.253360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ed1a-128f-4de4-94e9-50768b536a75 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.253869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ed1a-128f-4de4-94e9-50768b536a75 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.255803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 213/417] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:24.262024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.262649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.263137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.284361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:24.285901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:24.304572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:24.324504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 205/418] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:24.368069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 214/419] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:24.621496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:24.656764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:24.734877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bcd27-25c9-4b30-90ba-7ee79a2d34c4 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:24.780570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:24.781207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.781485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.781756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.802069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:24.807146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:24.808081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:24.808949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:24.813650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 206/420] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:24.815745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:24.847725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69960d90-09a2-4c91-865b-8f2e00a2ae55 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:24.851674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:24.931270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:24.932380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:24.932662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:24.932931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:24.959895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:24.965352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:24.966254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:24.967028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:24.969740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 215/421] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:24.976136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:24.991878 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.019074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa01c8d-3dc2-4988-a2f5-17f0aaefebc0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.090398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f441b-9c44-440f-844e-ee6ef1f23bc8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.125371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.125990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.126302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.127136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.128008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] RBAC: Adding query filter params (name=member) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:25.128341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.129738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.130022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.130830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.131498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.132011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.133092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:25.133598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.135131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.145864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 216/422] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:25.148541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 207/423] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:25.156981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.157127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.240861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.246777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.263681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdc0a5-250c-4767-877c-58d7d120c30f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.266497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad404ee-9bde-42c8-85e3-54d6b0e80d25 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.306474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.307087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.307365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.307638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.307967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.309445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.309926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.310848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.393245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.394937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbceb47-8936-4a76-8f56-4a306d5adf12 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.401221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.402679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b108-0652-40f8-9742-33216cb29c27 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.485754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.489914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.514343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 208/424] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:25.515884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40638453-1d6e-48a8-8d89-4e23cb17043b None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.516194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40638453-1d6e-48a8-8d89-4e23cb17043b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.516643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40638453-1d6e-48a8-8d89-4e23cb17043b None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.522456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 209/425] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:25.523634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40638453-1d6e-48a8-8d89-4e23cb17043b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.527494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 217/426] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:25.529775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.530130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.530548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.555182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:25.557183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:25.641157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 218/427] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:25.646191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.647067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40638453-1d6e-48a8-8d89-4e23cb17043b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.689511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.690252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.690595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.690936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.692159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.693720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.696496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 210/428] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:25.702488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.794558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36b77-195a-4a30-8dd2-32707d9fa508 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.795977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.821746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:25.834605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.835239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.835515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.835782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] PATH_INFO: `/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.837275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] RBAC: Authorizing `identity:get_project(project_id=c608b62797cb4b1aad77c875f6d5c5cb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.838972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.841169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 219/429] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/projects/c608b62797cb4b1aad77c875f6d5c5cb => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:25.847333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:25.911178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc5e4f-a2df-4eeb-acc0-92eb8b2f0669 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:25.965064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:25.965968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:25.966645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:25.967071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:25.968689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:25.970986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:25.973189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 211/430] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:25.978527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.066144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af3602-ce96-4b60-928c-9f4087d4453e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.101058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6e4f-a1ea-4968-819d-f7cbb49bce40 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.110328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.111784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.112132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.112497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.114165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:26.116458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:26.119147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 220/431] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:26.124319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.127352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:26.148513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.149414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.149526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.149704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.150500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:26.150813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:26.152333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:26.165492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 212/432] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:25 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:26.167545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.174741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:26.252992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c156f-5a00-4c9c-8b3a-98fc386595a8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.297216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.299052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.299580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.300068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.314719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:26.320374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:26.321304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 25 02:47:26.322086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:26.325092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 221/433] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:26.332717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.409210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfa88a-a4bb-452c-8c3c-2428d1ba57ff None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.454109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:26.459697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.459925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd0c4-6fb6-4f25-ae4b-c994f0517964 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.462167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.462167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.462167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.462167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:26.462167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:26.463363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7542c-4d60-4df8-89b6-08296a7bbc0e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:26.470945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 213/434] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=c608b62797cb4b1aad77c875f6d5c5cb&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 611 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:26.472677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2188d-43f9-4563-8dce-3b3d3a51a837 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.473035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2188d-43f9-4563-8dce-3b3d3a51a837 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.473385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2188d-43f9-4563-8dce-3b3d3a51a837 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.474998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 214/435] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:26.480901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85d24-737f-4dab-9066-f66eedc299bd None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.481266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85d24-737f-4dab-9066-f66eedc299bd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.481614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a85d24-737f-4dab-9066-f66eedc299bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.505794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f1a85d24-737f-4dab-9066-f66eedc299bd None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:26.507370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f1a85d24-737f-4dab-9066-f66eedc299bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:26.513250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.513982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.514359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.514704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.515472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] RBAC: Adding query filter params (name=member) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:26.515897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:26.518069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:26.528183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 222/436] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:26.534449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.568883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:26.600440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 215/437] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:26.604177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a8f0-2075-4e23-96d9-84b59b869597 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.604177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a8f0-2075-4e23-96d9-84b59b869597 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.604177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a8f0-2075-4e23-96d9-84b59b869597 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.604177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 216/438] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:26.609879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daea0-5386-4afd-af46-3913da458968 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.611358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daea0-5386-4afd-af46-3913da458968 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.611638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daea0-5386-4afd-af46-3913da458968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.635064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d95daea0-5386-4afd-af46-3913da458968 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:26.636336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d95daea0-5386-4afd-af46-3913da458968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:26.672624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436e889-96fb-4533-8b93-db23fd5bd581 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:26.719331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:26.720041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:26.721056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:26.721341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:26.732301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 217/439] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:26.756541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:26.802396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:26.804181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:26.886348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95daea0-5386-4afd-af46-3913da458968 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:26.916653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 223/440] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:26.925263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.012932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.020792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95daea0-5386-4afd-af46-3913da458968 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.029940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.059217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.061258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.061258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.061258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.062263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ffd67-1ad7-45fe-ab60-e5129079f3cd None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.072597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:27.077452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:27.078315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:27.079136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 218/441] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:27.080070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.110315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.111547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.111826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.112153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.113414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.114879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.116631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 224/442] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:26 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:27.118663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.198614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5fee1-d77a-422b-b74f-972eaae9a525 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.226818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b6ff9-a5cf-4b5e-895b-3d2a8b711d89 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.232164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.232801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.233085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.233374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.244113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:27.248893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:27.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:27.250696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 219/443] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:27.251390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.259095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.266424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.267603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.267923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.267923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.268954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:27.269256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.270609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.278317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.283603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 225/444] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:27.284476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.351670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66900c-dee0-4f65-8c92-4faeccabc837 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.383892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0770d90-3900-4f40-a59c-069303b82902 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.384619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.385248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.385533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.385809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.386936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.388428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.389993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 220/445] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:27.390756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.423278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.423969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.424284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.424559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.425218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:27.425517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.426869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.434809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 226/446] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:27.435583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.485848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.491581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.492224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d663e-4f85-4ed5-a05c-8d279e2e85d2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.524415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.525009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.525290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.525561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.537775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23095ba-80ee-45e1-a3b2-c5643b97706b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.576923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.577525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.577805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.578080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.578810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=5baef496d0274b11922ddd2613e3ff1a, scope.project.id=d5805663da164311887839f7a5cfa666) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:27.579123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.581112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.586857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 227/447] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=5baef496d0274b11922ddd2613e3ff1a => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:27.587629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.604302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.606013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.709144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2d1e-7dac-418d-84e3-5acb93fabac4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.714681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.723040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 221/448] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:27.730035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.731502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.751187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.752316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.752657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.752932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.833766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.835809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.853347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895150dd-a521-4d1a-9260-7bdfff5fa208 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:27.895573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:27.896252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:27.896530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:27.896802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] PATH_INFO: `/v3/projects/d5805663da164311887839f7a5cfa666` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:27.897925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] RBAC: Authorizing `identity:get_project(project_id=d5805663da164311887839f7a5cfa666)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:27.899388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:27.901209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 222/449] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/projects/d5805663da164311887839f7a5cfa666 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:27.906594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.959313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 228/450] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:27.964824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:27.978693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:27.981966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:28.016835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad7131-d367-463f-9280-a6c52cf08621 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:28.051147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:28.051766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:28.052043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:28.052352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] PATH_INFO: `/v3/groups/adaa3b34edb242588d15300aeec9da83` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:28.053449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] RBAC: Authorizing `identity:get_group(group_id=adaa3b34edb242588d15300aeec9da83)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:28.054882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:28.056619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 223/451] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/groups/adaa3b34edb242588d15300aeec9da83 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:28.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:28.068389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efa7a9-156a-4e29-8c98-4ed0001603eb None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:28.101481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:28.102103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:28.102379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:28.102647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] PATH_INFO: `/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:28.103756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] RBAC: Authorizing `identity:get_project(project_id=eabb40ce2ddb40b4bea318792fc09f8e)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:28.105205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:28.106787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 229/452] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:47:27 2022] GET /identity/v3/projects/eabb40ce2ddb40b4bea318792fc09f8e => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:28.112797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:28.162182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6290-e903-4f8c-96c2-8d653ab21f67 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:28.191221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:28.196662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:28.197338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:28.197487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:28.197609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:28.197802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:28.198553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] RBAC: Adding query filter params (group.id=adaa3b34edb242588d15300aeec9da83, role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=d5805663da164311887839f7a5cfa666) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:28.198874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:28.200422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:28.206250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 224/453] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:28 2022] GET /identity/v3/role_assignments?group.id=adaa3b34edb242588d15300aeec9da83&scope.project.id=d5805663da164311887839f7a5cfa666&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:28.214952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267375be-7919-4ece-9a63-3dcfdde748f3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:28.251270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:28.252856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:28.252998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:28.253194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] PATH_INFO: `/v3/groups/00c9fbf78e6144c797ac625539e7ec5d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:28.254415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] RBAC: Authorizing `identity:get_group(group_id=00c9fbf78e6144c797ac625539e7ec5d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:28.255877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c02f1-0528-42d4-9f7c-1de26a8e9872 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:28.257578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 230/454] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Aug 25 02:47:28 2022] GET /identity/v3/groups/00c9fbf78e6144c797ac625539e7ec5d => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:28.263602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:28.368539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16379c7a-68a4-4a0d-8315-f8a1cad4e73f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:28.402550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:28.403151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:28.403433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:28.403705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:28.404468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] RBAC: Adding query filter params (group.id=00c9fbf78e6144c797ac625539e7ec5d, role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=eabb40ce2ddb40b4bea318792fc09f8e) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:28.404778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:28.406274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136f6d-2f33-49a0-8d18-1e189105de3f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:28.412796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 225/455] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Aug 25 02:47:28 2022] GET /identity/v3/role_assignments?group.id=00c9fbf78e6144c797ac625539e7ec5d&scope.project.id=eabb40ce2ddb40b4bea318792fc09f8e&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:30.953829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a00f-d3a1-4b4e-b164-74b78de73bf7 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:30.955291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a00f-d3a1-4b4e-b164-74b78de73bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:30.955912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a00f-d3a1-4b4e-b164-74b78de73bf7 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:30.957761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 231/456] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:30.963394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577630cc-cb5a-4683-a448-fde31d442394 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:30.964253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577630cc-cb5a-4683-a448-fde31d442394 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:30.964507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7128-3191-4270-8263-75b5bb7f4367 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:30.964758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577630cc-cb5a-4683-a448-fde31d442394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:30.965233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7128-3191-4270-8263-75b5bb7f4367 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:30.965732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7128-3191-4270-8263-75b5bb7f4367 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:30.967488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 232/457] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:30.973073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ea88-b2c4-4ff6-9dde-65d148bc35aa None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:30.973357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ea88-b2c4-4ff6-9dde-65d148bc35aa None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:30.973622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ea88-b2c4-4ff6-9dde-65d148bc35aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:30.992343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-56e9ea88-b2c4-4ff6-9dde-65d148bc35aa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:30.993138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-577630cc-cb5a-4683-a448-fde31d442394 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:30.994118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-56e9ea88-b2c4-4ff6-9dde-65d148bc35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:30.994327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-577630cc-cb5a-4683-a448-fde31d442394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:31.081235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 233/458] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:31.088953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:31.107092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 226/459] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:31.270879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cf46-3629-42b8-abf9-dac849b6a85d None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.271525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cf46-3629-42b8-abf9-dac849b6a85d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.272002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545cf46-3629-42b8-abf9-dac849b6a85d None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.273811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 234/460] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:31.285773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.286067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.286348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.317176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:31.318387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:31.402435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 227/461] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:31.455297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545cf46-3629-42b8-abf9-dac849b6a85d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:31.513108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:31.584582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:31.623388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545cf46-3629-42b8-abf9-dac849b6a85d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:31.679702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda49332-c143-4b04-9d04-5e6a7ae9d819 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:31.684026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:31.684692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.684971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.685236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.702063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:31.703552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:31.706162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-74e12b1d-995c-4adb-985b-75cbff994117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:31.721116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:31.722383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 235/462] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63058-aaac-4ea3-9290-7c6d6f01fb13 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.723939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63058-aaac-4ea3-9290-7c6d6f01fb13 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.724328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63058-aaac-4ea3-9290-7c6d6f01fb13 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.728342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 236/463] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:31.728342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63058-aaac-4ea3-9290-7c6d6f01fb13 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:31.731053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:31.733428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.733428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.733428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.743996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:31.745514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:31.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:31.761988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:31.763519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:31.763519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.763519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:31.763519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 228/464] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:31.763519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:31.846567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63058-aaac-4ea3-9290-7c6d6f01fb13 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:31.870896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f139-82d0-4572-8452-2b01d9121a13 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:31.885314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:31.885956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.886233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.886509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.888446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:31.898254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:31.899734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:31.909396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f209c-461b-4344-ace1-213605678ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:31.910245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f209c-461b-4344-ace1-213605678ead None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.910697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f209c-461b-4344-ace1-213605678ead None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.911137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f209c-461b-4344-ace1-213605678ead None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.912006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f209c-461b-4344-ace1-213605678ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:31.912527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f209c-461b-4344-ace1-213605678ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7ac38634-745a-4992-a2eb-971a3e5a7e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:31.919803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:31.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:31.922447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:31.922447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:31.922447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:31.922447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 237/465] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:31.922447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:31.922717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f209c-461b-4344-ace1-213605678ead None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:31.922789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:31.922789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:31.936512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 229/466] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:31.936512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f209c-461b-4344-ace1-213605678ead None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:31.943658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:31.945155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:31.964078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.022759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 238/467] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:32.023612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.043338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f209c-461b-4344-ace1-213605678ead None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.082251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804cba-07de-408a-a094-7081fac152bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.082962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cba-07de-408a-a094-7081fac152bd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.083254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cba-07de-408a-a094-7081fac152bd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.083527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804cba-07de-408a-a094-7081fac152bd None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.084191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804cba-07de-408a-a094-7081fac152bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:32.084502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804cba-07de-408a-a094-7081fac152bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.085985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804cba-07de-408a-a094-7081fac152bd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.098959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 230/468] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:32.104263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804cba-07de-408a-a094-7081fac152bd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.139226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3482b5a-6d3a-42f8-8ca6-29f9a582392b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.155321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.173820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.178239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a883fa-92cf-44ec-a3ba-f6f8dbf4ffa2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.201263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 239/469] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:32.202425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9505-2d04-4a32-9c01-2e79d1f22f28 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.202715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9505-2d04-4a32-9c01-2e79d1f22f28 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.202990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9505-2d04-4a32-9c01-2e79d1f22f28 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.203734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.208908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 240/470] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:32.208908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b9505-2d04-4a32-9c01-2e79d1f22f28 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.220537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804cba-07de-408a-a094-7081fac152bd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.263742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.264697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.265233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.265514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.266278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.268356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.309245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b9505-2d04-4a32-9c01-2e79d1f22f28 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.341669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.342273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.342548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.342821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.343447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.345409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad8d7-c659-4557-b220-b572d7ab5d8f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.388992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 231/471] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:32 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:32.389790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.435230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 241/472] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:32 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:32.436791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.437138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.437629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.461178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:32.462572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:32.496780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.539829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c877115-0667-4f2d-90dd-770f44a7957c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.553443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 242/473] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:32.554368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.586184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.586802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.587545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.587828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.588546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.590304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.685566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62316653-8a32-46db-a6de-b972fd757a16 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.730117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.730777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.731056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:32.731352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:32.743996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 232/474] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:32 2022] POST /identity/v3/users => generated 248 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:32.750974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:32.752966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:32.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:32.772590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:32.773418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:32.773418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:32.773418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:32.773418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 243/475] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:32.778767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.787052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.951309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622176-d96a-4b62-a251-89fb96311260 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:32.965321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:32.986185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:32.995650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:32.996453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:32.996737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:33.000464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:33.000464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:33.000464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:33.000464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:33.016076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 233/476] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:33.021945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:33.088888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df976cc-cdc7-4a8d-9def-7228fa14214f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:33.100687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:33.144868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:33.145483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:33.145998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:33.146271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:33.147591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:33.149006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:33.149335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8fa70-2c97-44bf-869e-d1d1f61aa698 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:33.187592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:33.188758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.sql.core [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 02:47:33.188758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 25 02:47:33.188758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [parameters: {'id': '3dd0e23a435e4c14a5b0bfdf50730c7c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 25 02:47:33.188758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81785) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 02:47:33.205035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:33.205322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:33.205590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:33.206279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:33.207753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:47:33.215295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return ref.to_dict() Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:47:33.216234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application yield resource Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise self.value Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application session.commit() Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 02:47:33.217136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.flush() Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application compat.raise_( Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 02:47:33.218004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:47:33.218899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:47:33.219739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [parameters: {'id': '3dd0e23a435e4c14a5b0bfdf50730c7c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:33.222924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 25 02:47:33.224874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.226817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 244/477] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Aug 25 02:47:32 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:33.226817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:33.286889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 234/478] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:33 2022] POST /identity/v3/users => generated 249 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:47:33.333447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:33.360210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9d5d-c550-4f65-ad9b-930a2d50b629 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:33.407303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:33.408537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:33.408818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:33.409092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:33.425953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-b555dd20-ce3a-4fbd-8b0c-051d2cf41857 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:33.433408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:33.435552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:33.437512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 245/479] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:47:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:33.442355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:33.562020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d872fd-488f-4a60-bb46-16429707cb3f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:33.580211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:33.602327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:33.602938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:33.603216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:33.603488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:33.604221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:33.604531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:33.605968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e25a2-51b6-47c8-8a51-4cc510012790 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:33.615113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 235/480] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:34.848014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85543d8-e06a-4116-8af0-bc64011b7cb8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:34.848554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85543d8-e06a-4116-8af0-bc64011b7cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:34.848631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85543d8-e06a-4116-8af0-bc64011b7cb8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:34.850337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 246/481] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:34.856224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f21-ad71-4d35-9412-edcbd99827ff None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:34.856844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f21-ad71-4d35-9412-edcbd99827ff None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:34.857240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f21-ad71-4d35-9412-edcbd99827ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:34.882368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a4e02f21-ad71-4d35-9412-edcbd99827ff None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:34.883562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a4e02f21-ad71-4d35-9412-edcbd99827ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:34.922419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973e4cc-3aa5-4639-83a4-3dd3d77953eb None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:34.922791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973e4cc-3aa5-4639-83a4-3dd3d77953eb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:34.923520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973e4cc-3aa5-4639-83a4-3dd3d77953eb None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:34.928815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 247/482] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:34.930535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be2fd-7c2d-464c-81a0-3166e9d935f9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:34.930824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be2fd-7c2d-464c-81a0-3166e9d935f9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:34.931648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be2fd-7c2d-464c-81a0-3166e9d935f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:34.964384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4c7be2fd-7c2d-464c-81a0-3166e9d935f9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:34.965474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 236/483] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:34.965595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4c7be2fd-7c2d-464c-81a0-3166e9d935f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:35.074772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 248/484] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:35.176817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d5f8-2249-4ba6-b344-e1139d5d2f16 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.177161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d5f8-2249-4ba6-b344-e1139d5d2f16 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.177370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d5f8-2249-4ba6-b344-e1139d5d2f16 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.178980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 237/485] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:35.184190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.184452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.184722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.203073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:35.204347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:35.271149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:35.276733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 249/486] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:35.438221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d5f8-2249-4ba6-b344-e1139d5d2f16 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.446070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.578392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd47e32-c136-48aa-8678-e155fd4723c7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:35.621115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d5f8-2249-4ba6-b344-e1139d5d2f16 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:35.635722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:35.636712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.637185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.637656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.649632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:35.655383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:35.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.657425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:35.660705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 250/487] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:35.671479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:35.672324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.699334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:35.700472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.700818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.701156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.715591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:35.721063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:35.721953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:35.722745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 238/488] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:35.723407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.755785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:35.817612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513debd-393a-4375-a6d9-cc403b40d756 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:35.843066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad759d-de73-490d-ae53-ba0a337a081a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:35.855280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:35.855895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.856220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.856502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.857170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:35.857476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:35.858852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:35.867824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 251/489] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:35.868764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.891571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:35.892882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.893168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.893864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.904414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-fe71eeb3-1ddc-4198-a460-07e831012cad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:35.910346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:35.911254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:35.912060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 239/490] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:35.912769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0d972-576e-4a5d-9ee6-0fb067360c56 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:35.917978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0d972-576e-4a5d-9ee6-0fb067360c56 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:35.917978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0d972-576e-4a5d-9ee6-0fb067360c56 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:35.917978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 240/491] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:35.918243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0d972-576e-4a5d-9ee6-0fb067360c56 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:35.955034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:35.988816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d81996-c545-4a60-ad7f-6425ea6ffc8d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.012474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.027684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.028676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.029137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.029615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.030452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.030996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.031811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0d972-576e-4a5d-9ee6-0fb067360c56 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.033390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.043685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 252/492] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:36.044850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.076289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.077081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.077365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.077643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.089024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.090501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-59a05157-185c-4636-9756-9dea33a0e7a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:36.106296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:36.107245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:36.108218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:36.108218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.108218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:36.108218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 241/493] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:36.108715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.108946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.109221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.128606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:36.129802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:36.162054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb1033-804e-45a8-8f27-c1a66b22b1ff None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.197028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.197700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.197911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.198181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.198839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.199143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.208841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 242/494] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:36.208993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.209077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.210420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 253/495] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:36.210542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.222646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.258626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.310088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f1898-1990-4d11-a43b-39ec5d26460c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.319196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6200561-0bcd-48e5-b1a2-f71e15d96441 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.348669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.349298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.349580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.349853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.353016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.353619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.353901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.354175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.354817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.355125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.356587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d6061-fccc-443b-b299-2d3123f2d59e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.360338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.362188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.368295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 254/496] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:36.369567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a72d8-c39d-449c-acdd-97e43dca7aed None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.369856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a72d8-c39d-449c-acdd-97e43dca7aed None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.370132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a72d8-c39d-449c-acdd-97e43dca7aed None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.375410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 255/497] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:36.375410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a72d8-c39d-449c-acdd-97e43dca7aed None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:36.384300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:36.385188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:36.386052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:36.386052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.386052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:36.392250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 243/498] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:36.392250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.456732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.480092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a72d8-c39d-449c-acdd-97e43dca7aed None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.482405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.496128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea876a5-3fb1-4dbe-90fe-f23cc2317f39 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.515319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.515952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.516263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.516542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.526562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.528091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.529541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.530166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.530460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.530762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.538144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.539577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ef876abe-332b-4166-883a-8a316b6e71d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:36.542273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:36.543353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:36.544568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:36.544568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:36.544568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:36.544568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 256/499] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:36.544884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.545163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.545432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:36.552925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:36.553949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 244/500] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:36.554859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.564308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:36.565518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:36.632893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 257/501] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:36.633740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.669211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf593-ad90-403f-9b01-032534ce8800 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.701950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.715035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.715642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.715922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.716223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.717372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.717372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.718697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.731203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 245/502] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:36.731867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.746874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.763671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0911ec-aec7-4a17-adb8-c09797295873 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.800573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.801248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.801535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.801812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.812092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:36.817384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:36.818267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:36.819170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 258/503] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:36.819837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.839798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa7e43-8ad9-431a-9a3e-3e38c704a6b0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.876000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.876649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.876982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.877247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.877947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.878259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.879688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.891456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 246/504] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:36.892546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:36.941337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0360-1101-4cfe-8a6f-4c3854011371 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:36.945955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:36.974565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:36.975254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:36.975536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:36.975809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:36.976572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:36.976881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:36.978312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:36.999385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940147a-1227-4214-ae49-055dcb17a415 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.014334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 259/505] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:37.015680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.020971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.035400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.043901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.043901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.043901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.045126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.046755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:37.057786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:37.058853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 247/506] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:37.059777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.119516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bf8ce-af95-46b8-9c85-f63aca033d78 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.153171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.153778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.154053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.154323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.154981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:37.155292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.156694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.159257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7f44-16a7-4cdd-a799-74cffd2dbaf2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.161514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.166548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 260/507] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:37.167349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.193876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.194506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.194776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.195053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.202105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.203680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:37.215552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:37.216546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:37.217417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 248/508] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:37.218306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.235165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.270953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cfc37-802a-4b16-ae80-441189f84a7e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.304520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.305147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.305455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.305702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.316608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.318063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.320003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f41f73-bab9-4161-a816-85b5634304da None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:37.331948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:37.332865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:37.333712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:37.333712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.333712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:37.333712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 261/509] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:37.334262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.355906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.356541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.356816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.357085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.387955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.425393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.426890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.433997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869d7d6-1528-4443-a93a-f296a33f1ed8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.451457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.467643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.468279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.468552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.468820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.469565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:37.470404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.471786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.491054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 262/510] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:37.491824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.526386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 249/511] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:37.527157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.590714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.599857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be856261-37db-437c-8120-9cef4c2f472d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.629601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b814c-c33c-4e64-9ea1-509d1aafbff6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.636272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.636901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.637181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.637455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.647360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.648834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.662838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.663431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.663707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:37.663801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:37.664744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:37.665596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:37.665596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.665596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:37.665757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.665757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:37.665757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.666409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.669884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 263/512] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:37.670967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.700129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.703524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 250/513] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:37.704745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.775851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb64fd2-027c-464c-b4ea-c67213fa00ad None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.807392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a50821-d0ae-437f-8307-ac548847d442 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.814050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.814673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.814990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.815295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.816420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:37.816764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.818164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.830465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 264/514] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:37.830834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.831981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:37.842159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.846831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.846831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.846831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.911930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.913434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.918589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:37.933084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcefb74-5709-4650-9400-a132eeb0fed7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:37.966961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:37.967607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:37.967849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:37.968112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:37.978720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:37.980172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:37.994337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:37.995414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:37.996449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:37.996449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:37.996449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:37.996449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 265/515] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:37.996449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.021212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 251/516] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:38.021212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.042071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.099548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908a0b-a478-4769-af2b-c1ffe263e643 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.116026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e486ef-72e0-4598-b45e-ee92227ce4e7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.132922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.133526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.133802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.134069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.134701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:38.135002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.136374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.145467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.148803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 266/517] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:38.149395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.150172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.150785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.151052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.151319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.217605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.219080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.241365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.246166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3939bfc-14d8-4421-8c7b-ef09e45aa793 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.281253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.281896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.282142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.282462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.283055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.284526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.316977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 252/518] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:38.317811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.371202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 267/519] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:38 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:47:38.372021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.421670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df0de-d0c1-44b8-98c4-0bef6ae69e5f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.446788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.459007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.461658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.462273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.462555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.462832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.463493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:38.463803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.465371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.479278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 253/520] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:38.480172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.512781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6132e-2d9e-47ed-a49a-c883ae43ff56 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.562910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.564348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.564643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.565076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.578414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.580061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:38.595217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:38.600432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:38.601292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:38.601292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:38.601292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:38.601292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 268/521] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:38.601292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.604723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e8a-df15-4d38-9e03-20129e157799 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.651921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.652648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.653108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.653568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.665008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.667521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:38.687834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:38.689410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:38.690439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 254/522] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:38.691671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.733326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d7fa-3f28-4503-8a8b-6b3ed0b3cac1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.740279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.742750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:38.771554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.772194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.772478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.772749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.786006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.787460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.802781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb950-9aff-4cd1-8f62-0a69f625c101 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:38.802938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:38.803789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:38.804701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:38.804701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:38.804701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:38.808486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 269/523] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:38.809334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.841012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.841752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.842031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.842303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.850442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.851916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:38.877288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:38.878399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 255/524] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:38.879233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:38.917907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722b16-ca5e-40bb-ad4d-b26f0b5b0ea2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:38.959157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:38.959811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:38.960085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:38.960415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:38.961230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:38.961537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:38.963421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:38.978466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 270/525] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:38.981061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.001489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca2553-924e-4d6b-86bc-5bee3fad7997 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.003232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.005590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.037852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.038459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.038741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.039063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.039786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.040497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.042411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.067049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 256/526] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.067049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.102845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbe9b-4721-4e92-bf02-aa426bf679e1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.143102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.143712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.143991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.144295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.144941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.145246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.147067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.161084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 271/527] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.162023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.176071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f331813-c758-473a-a325-5dd18a0c82bc None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.217798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.218733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.219199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.219686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.220769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.221185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.223392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.250186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 257/528] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.250186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.258340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.278061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.283349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586cbd9-0e02-452f-b4a6-8a16e17de5bd None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.327124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.328223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.328534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.328812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.336238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.337696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:39.355993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:39.357092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:39.358149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:39.362235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 272/529] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:39.363424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.375614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa519-99f1-48d1-8e75-a83cda066ce5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.417230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.417886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.418167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.418447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.491132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8f73b-3749-4f4d-bccb-94f572ebe9c0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.497717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.499233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.524826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.531597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.532298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.532503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.532779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.539859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.541324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.543990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:39.554184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:39.555250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:39.556319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:39.560204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 273/530] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:39.560926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.638381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 258/531] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:39.639816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.677887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd36de-4426-46f2-8d77-a77a789eea7d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.715508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.716334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.716557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.717087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.718181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.718494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.720508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.736178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 274/532] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.736178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.785214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759c680-7490-4ed2-aa92-2af1e8ee6004 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.816293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.822619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:39.849731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.851383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.852098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.852609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.853616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.854456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.856512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.876687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d744-fe00-46bd-9b8a-ee4b24b5e5e6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:39.877494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 259/533] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.878824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:39.931616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:39.932307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:39.932589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:39.932857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:39.933693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:39.933844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:39.940183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:39.954367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 275/534] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:39.955316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.013353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea80c2-e788-483b-b94e-a70e47ea4ceb None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.058402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.059221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.059595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.059962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.060291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb50f8-921f-45d8-acfb-4eced29e59ba None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.074358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.076285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.087753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:40.100330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:40.101467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:40.102575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:40.102575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:40.102575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:40.103521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.104279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.108746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.108746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.108746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=ca96b68db5ab4c42ae4d34f6ce0a21d2) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:40.108746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.108746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.119949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 260/535] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:40.119949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.120230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 276/536] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:39 2022] GET /identity/v3/role_assignments?user.id=ca96b68db5ab4c42ae4d34f6ce0a21d2&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:40.120230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.155708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:40.231557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea372d-fa33-4bff-b1a9-a584013973f5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.259755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab1a2-9e0b-48a0-b171-0116a35e6747 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.271884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.272801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.273183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.273552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.274360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:40.274924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.276304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.290616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 277/537] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:40.296955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.303760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.304671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.305095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.305509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.306387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=57f08a89a0dd4befb153c2b38167b5c3) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:40.306874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.308543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d725fc-2f41-46d3-b5bf-ba89bb501a18 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.315013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 261/538] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/role_assignments?user.id=57f08a89a0dd4befb153c2b38167b5c3&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:40.358505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:40.401402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78ee7-199c-46d3-8218-260ebaefdc1d None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.402081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78ee7-199c-46d3-8218-260ebaefdc1d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.402357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78ee7-199c-46d3-8218-260ebaefdc1d None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.406112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 262/539] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:40.413569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.413878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.414121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.421850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6da1a7-dead-45f8-bf41-45208c9041ae None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.436893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:40.438288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:40.465916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.466567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.466915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.467191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.475472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.477039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:40.489853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:40.490770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:40.491632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 278/540] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:40.496991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.594591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 263/541] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:40.611686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e2aa-8a40-4b5d-a63d-65b5c54ff369 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.624541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:40.663523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.664166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.664446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.664719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.665416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:40.665722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.667160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.685398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 279/542] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:40.691299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:40.745743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:40.840062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99bdc6-18d5-4ef4-b9b9-3b8961d61f90 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:40.876678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001452-27af-4548-9b2f-60600c195787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:40.877291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001452-27af-4548-9b2f-60600c195787 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:40.877574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001452-27af-4548-9b2f-60600c195787 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:40.877894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001452-27af-4548-9b2f-60600c195787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:40.878631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82001452-27af-4548-9b2f-60600c195787 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=0c6b4345e6cd418f8b2a3b6e0b054ddb) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:40.878944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82001452-27af-4548-9b2f-60600c195787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:40.880502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82001452-27af-4548-9b2f-60600c195787 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:40.886507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 264/543] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/role_assignments?user.id=0c6b4345e6cd418f8b2a3b6e0b054ddb&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:40.982308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.092240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b083ce-ca80-4bf8-8d59-5c8ff0661e3b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:41.131079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:41.131641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:41.131915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:41.132203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:41.143309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:41.148028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:41.148955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:41.150131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 280/544] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:41.155217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001452-27af-4548-9b2f-60600c195787 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.249347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:41.258912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001452-27af-4548-9b2f-60600c195787 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:41.293489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:41.294100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:41.294375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:41.294644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:41.295304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:41.295859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:41.297262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:41.305836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 265/545] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:41.311395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.348791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:41.423677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc2295-c3c8-4b3e-a2cb-a6d262a98b66 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:41.459721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:41.460379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:41.460658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:41.460926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:41.471205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:41.472668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:41.487473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:41.488448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:41.489769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:41.489769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:41.489769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:41.489769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 281/546] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:41.494704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.600243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fc747-26b6-44a8-ab02-f3d237edd194 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:41.635420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:41.636033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:41.636374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:41.636643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:41.637286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:41.637588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:41.638958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:41.656549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 266/547] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:41.660470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.778497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:41.806362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf3d0-d248-4ab3-9192-2c2da32e075a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:41.856292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:41.857724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:41.858281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:41.858764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:41.871633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:41.873336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:41.888685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:41.889773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:41.890677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:41.893630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 282/548] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:41.899330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:41.931994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:42.086748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86098d5-395e-402f-9516-d377e47b84c3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:42.135959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:42.136612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.136888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.137156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.137888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:42.138190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:42.139620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732488f-39f8-40eb-99b6-981e3654b88a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:42.171947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 267/549] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:47:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:42.178535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:42.301039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a41c-1ebb-4217-bc24-c3ec2c05be2d None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.301389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a41c-1ebb-4217-bc24-c3ec2c05be2d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.301600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a41c-1ebb-4217-bc24-c3ec2c05be2d None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.303236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 268/550] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:42.309630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.309916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.310196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.332051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:42.333469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:42.343791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70227-5652-44b7-879c-d42b084e2f27 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:42.401829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:42.402453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.402733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.403131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.424561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:42.484550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:42.486429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62540a09-54f7-4033-89b8-d6a6d6021a8b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:42.505477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:42.521808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 269/551] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:42.632345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 283/552] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:42.640038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:42.706493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43726ce-829b-49cf-8566-2c02232faefe None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.707013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43726ce-829b-49cf-8566-2c02232faefe None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.707439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43726ce-829b-49cf-8566-2c02232faefe None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.709818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 284/553] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:42.715999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756fb6a-3500-4bda-a06e-dfc89180baaf None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.716471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756fb6a-3500-4bda-a06e-dfc89180baaf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.717176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756fb6a-3500-4bda-a06e-dfc89180baaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.743558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b756fb6a-3500-4bda-a06e-dfc89180baaf None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:42.748670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b756fb6a-3500-4bda-a06e-dfc89180baaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:42.784790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:42.785884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6cf7-72d0-49ee-89b6-d2ac67e147ed None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:42.831433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:42.832086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 285/554] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:42.832647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.833226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.833644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.833831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbec71-ccc8-4279-a7a2-c12324f63197 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.834592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbec71-ccc8-4279-a7a2-c12324f63197 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.835517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbec71-ccc8-4279-a7a2-c12324f63197 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.841735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 286/555] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:42.849335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:42.849786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:42.851028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:42.854708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:42.856428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:42.883416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:42.884684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:42.885762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:42.885762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:42.885762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:42.890019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 270/556] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:42.891208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:42.893488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:42.896528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:42.969926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:42.976469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 287/557] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:42.978480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.025173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35d67-0ea5-425b-b7eb-702f5109b73f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.063899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.064638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.064983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.065324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.066041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.066627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.068996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.082602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 271/558] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.088101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.167805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60930f-57d8-4dad-9dec-fa15938f00c6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.195468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5bdf3-b087-4f25-8cf5-d775f3902b29 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.196416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.211041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.216579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.217392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.217819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.218234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.229972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:43.233410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.234113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.234448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.234777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:43.236825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:43.240725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 288/559] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:43.241564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.247560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.249398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:43.262162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:43.263813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:43.264980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:43.268176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 272/560] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:43.269119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.361111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7243c-e167-4165-917d-7dedf859d97f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.371420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d112bc9-e83f-4508-b52d-b0cda686332e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.397442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.398071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.398346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.398612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.406533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.407643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.407917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.408300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.408964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.409087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:43.409266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.410625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:43.413753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:43.414753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 289/561] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:43.415579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.420711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 273/562] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.421444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.441446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.455315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.519291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ff843-bad3-4d91-bd16-0979d207e346 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.519933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999221c-f6dc-4d9f-a305-34bb63d56918 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.553549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.553918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.554149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.554425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.554520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.554694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.554786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.555064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.555375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.555678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.557136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de99fe-20c1-4a04-9aba-bd92101552e8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.565038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:43.568582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 290/563] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.569745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af15d4-8e9f-45ae-92b7-674de31ddbf4 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:43.569950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:43.570806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:43.571640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:43.572778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af15d4-8e9f-45ae-92b7-674de31ddbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.572778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af15d4-8e9f-45ae-92b7-674de31ddbf4 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.572906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 274/564] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:43.572906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.574000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 291/565] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:43.574833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af15d4-8e9f-45ae-92b7-674de31ddbf4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.667323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.679991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.686240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea67a27-8e83-42ae-bbe0-716c7254142e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.687945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af15d4-8e9f-45ae-92b7-674de31ddbf4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.719208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.720443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.720714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.720983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.721640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.721942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.722991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.723326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.723613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.723887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.724170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.731004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 275/566] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.731778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.738381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.739808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:43.754654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:43.755767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:43.756713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:43.756713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:43.756713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:43.756713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 292/567] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:43.756713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.834784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d790f-7fd8-485c-8eca-4f30e6a28349 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.854822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ac56-2d04-4cf7-bf0d-eed685c7f95f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:43.871076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.871687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.871971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.872290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.873021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=a8798813bd1e461d81faf73696f62a16) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.873332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.874777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835f6e-31e4-45ca-9bc0-35907d753d39 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.880938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 276/568] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/role_assignments?user.id=a8798813bd1e461d81faf73696f62a16&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.881909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.882197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.882474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.901219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:43.901468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.902497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:43.906912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:43.907846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:43.908364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:43.908840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:43.909701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:43.910193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:43.911795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:43.919649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:43.926782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 293/569] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:43.926782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:43.996206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 277/570] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:43.996206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.064639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a348c7-6b9a-48f5-999c-202db83543e4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.103969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.104838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.105201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.105531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.116478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.117926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.118175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b4550-a946-4ce7-ad74-2f1f6a83f080 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:44.134049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:44.135035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:44.135924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:44.135924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:44.135924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:44.139805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 294/571] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:44.140692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.154817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.155527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.155810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.156080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.157590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:44.158019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.160421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.174556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 278/572] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:44.175464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.190147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.193428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a0f9-6af0-4e46-985c-1d9a6228507f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.288377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.289169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.289454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.289727 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.302860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.304363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.320410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148e92a-cf64-46a2-9320-277b0e7fbb3f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:44.321607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:44.322540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:44.324130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:44.324130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:44.324130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:44.324130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 295/573] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:44.324130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.363822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.364516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.364794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.365067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.365820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:44.366126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.367525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.381322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 279/574] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:44.382173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.445653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.450855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0d981-2fc2-4801-8a4e-9904924c02a9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.472420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.495428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.496545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.496833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.499706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.508038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.509393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a29-2339-4314-a716-59e2811354d6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.509696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:44.531495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:44.533394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:44.534345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 296/575] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:44.534934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.555426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.556517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.557076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.557358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.578555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:44.585128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:44.586087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:44.586928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 280/576] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:44.592161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.689616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127651a1-5dd2-496a-88ae-5d427cf3b030 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.731296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d9ee-9e63-46aa-8159-765d3ea53740 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.754188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.755201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.755691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.756986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.757882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:44.758635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.760986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.774343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.775161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.775534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.776129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.779690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 297/577] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:44.780952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.788027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.789862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:44.809674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:44.810757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:44.811726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:44.814500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:44.819114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 281/578] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:44.820186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.913028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4bc3-7133-438a-8902-b79307973f67 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.942358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb743f-3700-4429-9618-aa0493677544 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:44.957896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.958667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.958958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.959238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.959978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:44.960341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.961777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:44.979634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:44.980296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:44.980580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:44.980858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:44.981542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 298/579] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:44.982200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:44.987576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:44.987576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:44.988654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.000301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 282/580] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.001930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.059856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.064783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.088412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd185f-470d-4da7-a8f1-be07e382854c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.112314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2dea-f334-48bd-9c84-2fed9a8ec711 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.122603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.123233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.123533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.123791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.132060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.133535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:45.145783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:45.146747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 299/581] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:45.147804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.155557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.156196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.156486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.156759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.158017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:45.158326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.159783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.178247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 283/582] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.178247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.267213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b74c1-c225-4759-8f56-6e089450151e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.289387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a00dc-1bbd-47f2-b3e3-a4da11eb0553 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.303460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.305357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.305963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.306351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.306674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.309700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.318271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.319719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.332302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.332942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.333230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.333507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:45.335030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:45.336001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:45.337580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:45.337580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.337580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:45.337580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 300/583] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:45.337580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.350081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.351591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:45.366616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:45.367607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:45.368555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:45.368555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.368555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:45.368555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 284/584] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:45.368555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.452640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2be81b-e12e-43b9-8576-13e95bf8e3f2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.491002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.491968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.492456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.492677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf11b34-3c76-4c4d-a9ce-306c065cd11f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.493110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.494042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:45.494500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.496079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.515129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 301/585] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.515972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.536816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.538119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.538567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.539026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.553269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.555707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.578284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:45.579423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:45.580719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:45.581630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:45.581630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.581630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:45.581803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 285/586] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:45.582506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.603766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.640174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc724e33-14f9-478c-80d0-d09a890ad663 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.686261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.687035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.687317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.687901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.689706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:45.690025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.692160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.693912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a5a51-c512-4d0e-8c76-32954558a37c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.706060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 302/587] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.711390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.751072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.752443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.752689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.752965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.754256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:45.754256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.755317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.772369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 286/588] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.772369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.834867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8a5a-c1ed-4a67-85e0-dea6396a48ed None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.842071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.858768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:45.871132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.871769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.872049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.874089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.884022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416e07e-5bb6-40a7-b1e1-cf0df90f807e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:45.885419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.886889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:45.901463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:45.902655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:45.903623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:45.903623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:45.903623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:45.903623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 303/589] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:45.903623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:45.923823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:45.924124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:45.924426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:45.924690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:45.925331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:45.925631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:45.927054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:45.939743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 287/590] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:45.940637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.011735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f30cf-7d2c-4c81-a13a-352fc14fcedc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.043326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159fb91-7329-4947-a85c-0f0865beb3de None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.047690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.048309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.048587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.048864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.055576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.057058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-efa1a2c7-acda-477a-99d6-67fd624227ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:46.069716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:46.070761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:46.071685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 304/591] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:46.072209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3587a5-c441-43a4-ba5e-6e687dee867e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.072378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3587a5-c441-43a4-ba5e-6e687dee867e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.072671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3587a5-c441-43a4-ba5e-6e687dee867e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.073855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 305/592] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:46.074554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3587a5-c441-43a4-ba5e-6e687dee867e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.083443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.084075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.084375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.084644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.086608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.092108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.093565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.095784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:46.105527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:46.106532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 288/593] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:46.108257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.184802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3587a5-c441-43a4-ba5e-6e687dee867e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.208245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1b56f-9b45-41ca-a622-44fe8d51fcaa None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.218631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.219231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.220169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.220499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.221143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:46.221537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.222801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.234311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 306/594] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:46.235248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.242158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.242785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.243065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.243338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.249964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.252101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-7942a580-6791-43b2-89f9-bb7986ee1735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:46.263496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:46.264563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 289/595] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.271189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.271873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.287409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:46.288649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:46.320297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.337632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.342853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b73df-16f1-49c5-b6f4-154ec5dde095 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.355647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 290/596] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:46.356501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.381886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.382591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.382825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.383095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.383784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:46.384088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.385535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.402619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 307/597] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:46.402619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.464039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec6bb-8f76-4677-8b2a-77e520d8b9d7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.497863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.498472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.498755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.499034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.499773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:46.500082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.502285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.508800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be55d5-c7c3-4569-8fee-1591d957e887 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.523886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 291/598] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:46.523886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.549585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.550299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.550654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.550995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.557806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.559358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.561160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.567628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:46.572215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:46.576652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 308/599] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:46.577817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.629806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70eb7d-c6e1-47b8-b457-aa51f1fa2bea None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.664686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.665295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.665573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.665843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.666527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:46.666834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.668280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.680112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 292/600] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:46.680925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.683049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228c9b-0f39-46e4-b55a-053936fb9f73 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.720113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.720797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.721073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.721349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.737570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=867d21f5f80d4cc48cc53e0b89c7f2f8, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.739019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.790949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7bcb2-a702-4cac-8057-4bc9c5be113d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.792447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.805181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 309/601] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:46 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:46.806002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.838598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.839240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.839512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.839789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.851430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.852965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:46.869309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:46.870392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:46.871289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:46.871289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:46.871289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:46.871289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 293/602] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:46.871525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:46.880705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:46.943733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733c7ff-0994-486b-b475-0a8fa217f61e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:46.989168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:46.989877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:46.990298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:46.990584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:46.991339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:46.991789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:46.993328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:46.998738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9143-0797-42b7-b7ed-887783027eab None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.006370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 310/603] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:47.011198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.042882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.043612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.043966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.044342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.055443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:47.060811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:47.061924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 294/604] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:47.062801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.081078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.132355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff8d84-4d78-4fc7-af8c-98c82dde7656 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.141344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.168191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.168857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.169146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.169418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/57f08a89a0dd4befb153c2b38167b5c3/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.177229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e16504-f405-4b00-bcd4-1bc681fed42d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.186293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=57f08a89a0dd4befb153c2b38167b5c3, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.187743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.217979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.218772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.219204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.220467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.221496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:47.221975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.223793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.238945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 295/605] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:47.239764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.253600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 311/606] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:47 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/57f08a89a0dd4befb153c2b38167b5c3/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:47.253600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.342540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.370893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566e0c-7ef8-46b9-aeb5-13c27ef4fdc8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.407336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6af5d-d2fc-4844-8220-40373c2942df None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.420239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.421827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.422583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.423157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/ca96b68db5ab4c42ae4d34f6ce0a21d2/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.445184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=ca96b68db5ab4c42ae4d34f6ce0a21d2, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.446651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.454149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.454767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.455043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.455316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.455984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:47.456325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.457679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.466511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 312/607] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:47.467297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.490796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.522426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 296/608] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:47 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/ca96b68db5ab4c42ae4d34f6ce0a21d2/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:47.523334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.656116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f924b4-dd9c-47ef-bf7b-4edbb7916dc4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.662271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e772f70-bd0c-473f-839f-43444cf60033 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.695753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.699736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.700408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.700694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.700972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.713701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.715183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:47.732185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:47.733236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:47.734217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:47.734217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:47.734217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:47.734217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 297/609] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:47.739960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.746339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.746997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.747285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.747555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.756032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.757518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:47.769933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:47.770858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:47.771746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:47.775664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 313/610] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:47.777067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.855441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccda0c6-5dd5-435b-ab78-11f150f3b5a9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.869640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:47.893398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ad4e-1cbf-4f89-967f-af4b0d1c5ad5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:47.895219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.895862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.896261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.896547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.897211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:47.897520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.898924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.911598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 298/611] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:47.927536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.935136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:47.935825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:47.936205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:47.936553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:47.937316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:47.937694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:47.940229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:47.958054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 314/612] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:47.959337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:47.981305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:48.067462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9b3bb-a6e2-4682-83d3-3ef54455738c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:48.071010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6d834-dd87-4603-adab-5d098154237f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:48.107174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:48.108249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:48.108753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:48.109285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/0c6b4345e6cd418f8b2a3b6e0b054ddb/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:48.124512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:48.129024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=0c6b4345e6cd418f8b2a3b6e0b054ddb, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:48.131251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:48.149798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:48.150695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:48.151179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:48.151670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:48.159954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:48.161795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:48.174628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:48.175576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:48.176645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:48.183317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 299/613] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:48.185175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:48.221479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 315/614] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:47 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/0c6b4345e6cd418f8b2a3b6e0b054ddb/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:47:48.337568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:48.345808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24bf53-18d6-4128-8e9f-821a11ed253a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:48.390045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:48.391077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:48.391729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:48.392474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:48.393432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:48.393945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:48.395598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:48.421637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 300/615] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:47:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:48.428877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:48.557446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663092-60d6-499b-9279-227d3950f05a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:48.609864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:48.610518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:48.610817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:48.611056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:48.625267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:48.627291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:48.648509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:48.649510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:48.651045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:48.651045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:48.651045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:48.651045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 316/616] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:48.655820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:48.773879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992098e-681d-4386-980c-18a2c03fb076 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:48.809049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:48.809581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:48.809867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:48.810146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:48.810799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:48.811109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:48.812552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:48.825153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 301/617] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:48.831317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:48.910403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:48.978386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e32d-2fa2-4dff-b45e-a83c15c1abb9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:49.016042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:49.017701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.017995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.018281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.025222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:49.026655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-a441945b-ca4a-4512-abbe-b3c004a76c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:49.038656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:49.039981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:49.040904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 317/618] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:49.049364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:49.055793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac0298-f4f8-47b8-b7ab-5b6d0588b4ff None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.056450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac0298-f4f8-47b8-b7ab-5b6d0588b4ff None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.056922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac0298-f4f8-47b8-b7ab-5b6d0588b4ff None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.058689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 318/619] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:49.067683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b2c6-58e1-49dd-8a39-5332529b8b4f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.068284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b2c6-58e1-49dd-8a39-5332529b8b4f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.069368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79b2c6-58e1-49dd-8a39-5332529b8b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.080497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:49.117303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6e79b2c6-58e1-49dd-8a39-5332529b8b4f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:49.118555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6e79b2c6-58e1-49dd-8a39-5332529b8b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:49.170056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29049-8458-445b-ad9d-665aef3a9037 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:49.208717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:49.209333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.209618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.209886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.211111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:49.211430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:49.213206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:49.227773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 302/620] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:49.233360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:49.254551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 319/621] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:49.325037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:49.350144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea7df-0e63-44c4-bdaa-df91ea5c8370 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:49.385690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:49.386351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.386629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.386903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/a8798813bd1e461d81faf73696f62a16/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.402003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=a8798813bd1e461d81faf73696f62a16, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:49.403464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:49.463057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 303/622] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:47:49 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/a8798813bd1e461d81faf73696f62a16/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:47:49.621283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5550f5a-3cf1-4ac7-a0b0-ba9b29636baf None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.628733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5550f5a-3cf1-4ac7-a0b0-ba9b29636baf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.628733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5550f5a-3cf1-4ac7-a0b0-ba9b29636baf None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.628733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 320/623] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:49.628978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:49.629898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.630182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.630455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.650375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:49.651701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:49.683443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:49.753586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 321/624] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:49.786664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dade349-12a9-4705-adcb-97b95abec416 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:49.813247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:49.824082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:49.824939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.825399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.829248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.838140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-9a5cadb3-a357-4f2e-a20c-c94e853c9176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:49.843422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:49.845203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:49.846473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:49.852388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 304/625] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:49.855114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:49.966190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df558c-e267-455d-86f5-6e40e7fd5d0a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:49.989738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807e99-fd6f-400b-944c-cf58c1848487 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.990031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807e99-fd6f-400b-944c-cf58c1848487 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.990329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807e99-fd6f-400b-944c-cf58c1848487 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:49.993504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 305/626] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:47:49.998756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:49.999265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:49.999495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.004659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.005611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.006080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.006619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.007466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:50.008033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.009640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.023685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 322/627] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:50.025184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.033348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:50.034633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:50.053480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.133188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49b9f-1281-446f-95df-2f627bd851d5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.157779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 306/628] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:50.158650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.172052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.173055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.173751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.174044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.187763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.190736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:50.209268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:50.210247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:50.211131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:50.211131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:50.211131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:50.212236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 323/629] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:50.217704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.304938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.331116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf8922-27fc-44f0-8fce-4bb8b06bbb8c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.368598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.369669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.370171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.370656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.371515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:50.372078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.373784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.386084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e1cde-74ab-4537-b94b-976a01af123a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.393899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 324/630] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:50.396766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.424557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.465255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.468915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.469236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.469518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.508759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:50.513654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:50.514696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:50.516661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 307/631] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:50.523266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.544342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbe9fd-73d4-40c5-80a0-24622290949e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.589987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.593769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.593769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.593769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.596261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.602286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.604409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:50.621320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:50.623150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:50.624116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:50.632415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 325/632] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:50.633197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.664114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cae715-b8f9-416f-8be8-dead9b3f9271 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.713914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.714535 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.714821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.715099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.716272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:50.716538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.718250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.733799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 308/633] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:50.735805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.775577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.778205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff36f50-f080-479a-a996-b9132b9d90aa None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.815222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.815855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.816138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.816458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.827392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:50.833128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:50.834163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:50.835009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:50.839076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 326/634] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:50.839939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.854329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851998-5cd3-4b72-aa20-cb597f79eba3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.894798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.895416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.895698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.895973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:50.896861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:50.897172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:50.898634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:50.900352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:50.924569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 309/635] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:50.927313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:50.959170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eec5e-5627-44d3-bc48-ed5e572ae9d1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:50.993670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:50.994231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:50.994514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:50.994792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.005698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.007162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6efa009b-615d-484b-95b0-e01af316dcaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:51.021882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:51.023035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:51.024098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:51.024098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.024098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:51.024098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 327/636] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:51.024451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc284ec5-3a87-4998-ab9d-cc29c50fdd49 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.024735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc284ec5-3a87-4998-ab9d-cc29c50fdd49 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.025065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc284ec5-3a87-4998-ab9d-cc29c50fdd49 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.026447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 328/637] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:51.027063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc284ec5-3a87-4998-ab9d-cc29c50fdd49 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.038751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.046595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4bb3-56da-4573-95ee-39aa5623f7b8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.084339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.085075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.085458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.085944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.086735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:51.087177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.088723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.102399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 310/638] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:51.102399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.137818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc284ec5-3a87-4998-ab9d-cc29c50fdd49 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.147512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.173450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.174198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.174534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.174885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.209564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da4-2ed9-46d8-956e-6625c0b0efaa None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.244661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.245189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.245473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.245747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.246389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:51.246695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.248074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d978-67ce-4945-8753-15d693a42c96 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.253876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.255446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.265122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 311/639] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:51.265122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.265122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.265122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.281457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:51.282801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:51.294271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.356630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 312/640] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:51.356630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.371480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 329/641] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:51.372882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.390297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.463506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aad7ff-a2da-4dbc-92d4-a759bbd8f3a7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.499872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.500814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.501108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.501392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.512499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf6ef4-d410-4b09-96eb-8c74be02dd7f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.512692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.513982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.524504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:51.535427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:51.538554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:51.539635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:51.539635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.539635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:51.539635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 313/642] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:51.539635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.556719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.557381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.557705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.558020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.566379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.568669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:51.583250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:51.584340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:51.585278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 330/643] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:51.585882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.663018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933274-3b26-400b-b6b7-79c38495cc84 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.663910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.700650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.701296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.701577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.701854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.702177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262abc7-a9cd-46e7-9529-0494bad0f5d0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.714014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.715488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:51.730083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:51.731064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:51.732106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:51.732106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:51.732106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:51.732106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 314/644] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:51.732106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.753042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.753603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.753952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.754270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.754962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:51.755312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.756771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.769861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 331/645] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:51.770728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.786265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.835871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918512f6-5b51-424b-b5cf-86edff25a054 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.868983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.869588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.869870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.870145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.870864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:51.871171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:51.872503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e4ded-317c-4ba6-8284-f6b139da283f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:51.872686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:51.889760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 315/646] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:51.890612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:51.900063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:51.908294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:51.908948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.909268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.909584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.919824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-cec1504e-a84c-4024-a01b-1d1f4d4341f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:51.926494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:51.927565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:51.928479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 332/647] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:51.929963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd77c-f1c0-42eb-87f8-82d766c4ca1c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:51.930285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd77c-f1c0-42eb-87f8-82d766c4ca1c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:51.930599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd77c-f1c0-42eb-87f8-82d766c4ca1c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:51.931886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 333/648] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:51.932697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffd77c-f1c0-42eb-87f8-82d766c4ca1c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.000010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aa2d3-f4ab-4618-b56b-2ec2eabda333 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.009541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.033465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffd77c-f1c0-42eb-87f8-82d766c4ca1c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.033786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.034387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.034660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.038508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.038508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:52.038508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.038508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.048972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 316/649] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:52.049902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.069961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.070580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.070854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.071124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] PATH_INFO: `/v3/users/glance` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.078014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.079502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:52.090661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:52.091786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 334/650] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:52.092834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.140161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.152629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256327fb-a2ee-4a9b-85e8-f3aa68869cbd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.186335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.186935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.187214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.187493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.190764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad23d3-d0e3-43a9-88c1-85290a3218be None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.221822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.223833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.224466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.224745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.225017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.225672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:52.225973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.227322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4187a6-d10a-4f73-bc25-f7de11ef9bcb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.235586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 335/651] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:52.236922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.237200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.237618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.255228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:52.256450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:52.256948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.258411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.321955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 336/652] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:52.322555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.366338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.367690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 317/653] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:52.368504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.428446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326e187-6da4-4400-808c-9d991d62f52e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.440425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.464607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.465358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.465640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.465905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.473586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.475010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.475131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01df26-3be9-45c8-a501-4539ffa345c2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:52.487367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:52.488312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:52.489254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 337/654] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:52.489763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.514416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.515015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.515294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.515569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.516404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:52.516717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.518140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.536264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 318/655] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:52.537072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.597003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.600023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d99f-5686-4dc8-b043-7f89c78c6938 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.633463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.634092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.636567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.636567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.643574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1770-8751-44d8-85a4-b171a56e38ab None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.644791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.646222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:52.661699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:52.666233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:52.667300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:52.667300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:52.667300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:52.667300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 338/656] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:52.667300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.672808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.681533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.682187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.682528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.682801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.692878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.694315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:52.709084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:52.710098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:52.710980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:52.710980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:52.710980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:52.710980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 319/657] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:52.710980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.769431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242084e-6a85-46a8-b6a6-7174dc37b046 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.803936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.804557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.804833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.805099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.805783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:52.806082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.807494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.811204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cfcdb-28f4-416e-9351-d549b6ee8136 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.819543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 339/658] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:52.820370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:52.836966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.853919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.853919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.853919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.853919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.897375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:52.919470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:52.921217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:52.925318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fc4fd-2f1a-41c3-ad45-e60d87493204 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:52.961356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:52.961984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:52.962258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:52.962530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:52.973284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:52.977976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:52.979084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 340/659] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:52.980234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.026216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 320/660] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:53.027049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.049690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.079023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775072-a84c-4494-aab7-3d3f77b434b7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.103104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.116044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.116044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.116044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.116044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.116415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.116415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.117609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.123749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ceae1-65db-42e4-a9ed-23b7f475dc35 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.129731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 341/661] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.130589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.159566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.160196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.160481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.160754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.161393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.161699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.163078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.174492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 321/662] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.175575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.230354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091f262-d120-4b15-80dd-e81f75f4cae0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.259651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.264474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.265069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.265347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.265651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.270490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.270490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.270490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.273968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c954-c750-406d-890a-0c7c3942ee46 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.274691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 342/663] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.275463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.329565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.330199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.330482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.330763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.331457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.331865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.333322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.339589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.343896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 322/664] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.344851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.393110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895b3d7-77e1-4b6e-9cad-10840ac5bed0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.438782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.439207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.439601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.439872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.453147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.455433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.465550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458608-1901-4b3c-95d3-d86f7e8e38c7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:53.471420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:53.472415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:53.473316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:53.473316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:53.473316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:53.473692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 343/665] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:53.474482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.506858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.508685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.508685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.508685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.516869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.518432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.527917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:53.529913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:53.530838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:53.532038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 323/666] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:53.532685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.587044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3c26d-6c66-4f73-beef-63541bc943f3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.599565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.625324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.626777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.627098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.627374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.634289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.635719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.645810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22f9b-137b-4ff9-8a2a-4a9692f6d069 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:53.647591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:53.648492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:53.649404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 344/667] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:53.651270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.687145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.688409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.688793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.688974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.702175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.703696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:53.718162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:53.719211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:53.720119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:53.720119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:53.720119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:53.720119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 324/668] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:53.720371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.763561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323caab-f73c-4fc6-9750-b6af3e63964e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.774004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.800982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.801600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.801891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.802165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.803540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.803540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.804718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.816755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 345/669] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.821191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.830072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a476-9fb8-4643-9fe9-7644ac559dc7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.846806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:53.867588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.868216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.868502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.868767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.869489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.869794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.871215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.889864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 325/670] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.891520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:53.941959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b00b9e-240a-4ac4-8610-8785a26e238c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:53.978171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:53.978776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:53.979056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:53.979326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:53.980015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:53.980348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:53.981839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:53.994463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 346/671] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:53.995505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.012132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f77d-7d2b-453e-84bc-fcc1d9f733bd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.029454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.049224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.049836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.050115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.050386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.051026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:54.051335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.052775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.065545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 326/672] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:54.066362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.097651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.108134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cdee7-819c-4b43-ad5b-ae8b6f16dbdb None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.150222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.151413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.151865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.152140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.160531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.162828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.169542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a67b-cf2f-4b1b-8728-88bdc5bc23d3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:54.174711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:54.175627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 347/673] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:54.181648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.211437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.212051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.212387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.212638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.275343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.285025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.286493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.287449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0de11-f658-48ed-8dd9-e440127248fc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.319358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.325286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.325286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.325491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.325732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.326455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=57f08a89a0dd4befb153c2b38167b5c3) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:54.326762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.328275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.340825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 348/674] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/role_assignments?user.id=57f08a89a0dd4befb153c2b38167b5c3&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:54.342199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.411595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 327/675] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:54.412849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.481115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948d4d7-dd47-4691-b3bc-d593f1e0a69f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.527730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.527730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.527730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.528113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.538373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.539812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:54.553398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:54.554439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:47:54.555451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:54.557994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 349/676] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:54.558983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.559264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f7b36-e58b-40d6-9b85-f5d0f37f5c04 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.577189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.594975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.605785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.606403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.606745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.607024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.608081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:54.608446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.611074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.628133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 328/677] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:54.629067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.677239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822778f7-d64e-4f7f-a578-3e96c6604e0e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.711039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.712060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.712576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.713060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.724950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.726757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:54.744168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:54.745136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:54.746030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:54.746030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:54.746030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:54.748199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 350/678] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:54.749210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.752957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbba4b4-8191-4a8e-b4cc-1e649b34ae98 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.802338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.803091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.803387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.803787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.804581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:54.804893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.806652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.825604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.833379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 329/679] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:54.836887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.875648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8d63-3688-4843-b8ab-7edadd257b03 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:54.893808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:54.927724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:54.928679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:54.929026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:54.929407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:54.930228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=ca96b68db5ab4c42ae4d34f6ce0a21d2) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:54.930711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:54.932075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:54.938912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 351/680] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/role_assignments?user.id=ca96b68db5ab4c42ae4d34f6ce0a21d2&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:54.939895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:54.963639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db70da5-1f6f-4a7c-81c0-9082feb14db7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.009282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.010506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.010991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.011962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.012909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:55.013408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.015032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.032187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 330/681] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:55.038908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.119991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23245b1d-ec1c-4e8f-928b-7e46271e23f6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.150102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:55.161485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.162658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.162772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.163318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.166635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac843a-2798-4b0b-a326-0525e84183ea None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.181177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:55.214821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.215462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.215743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.216022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.226947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.228740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:55.243915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:55.245035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:55.245945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 331/682] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:55.253418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.253576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.255132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289d850-e3d5-460d-8d5d-d011d9506e99 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.386030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 352/683] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:47:55.387215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d751f-0940-4f5d-b859-560d7c938b9b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.387354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b82830-3ca4-41a9-aa9d-6e94f172dbe0 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.387582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b82830-3ca4-41a9-aa9d-6e94f172dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.387853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b82830-3ca4-41a9-aa9d-6e94f172dbe0 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.389594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 353/684] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:55.391163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b82830-3ca4-41a9-aa9d-6e94f172dbe0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.426401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:55.431066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.431919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.432433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.432933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.433839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:55.434376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.436247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbe20c-eee0-42c1-80ff-ba510f3489e9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.449672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 332/685] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:55.451386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.451880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.452413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.476975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:55.479686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:55.511944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:55.513119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b82830-3ca4-41a9-aa9d-6e94f172dbe0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.551928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.552639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.552920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.553197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.564628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.566290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.576234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 333/686] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:55.577210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:55.592852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:55.599222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:55.600376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:55.600376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:55.600376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:55.600376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 354/687] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:55.603978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.752594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:55.773272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbee275-58b8-41c1-b78a-2608424b5e69 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.788216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202de15-de2a-4132-9ff5-860f3b37496c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:55.840799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.841412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.841695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.841971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.842705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=0c6b4345e6cd418f8b2a3b6e0b054ddb) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:55.843017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.845150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.858754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 334/688] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/role_assignments?user.id=0c6b4345e6cd418f8b2a3b6e0b054ddb&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:55.864336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:55.864972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:55.865295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:55.865580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:55.866260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:55.866574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:55.867995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:55.894752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 355/689] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:55.900550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.916840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:55.952544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.072810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc9624-6a95-4d2c-bfe4-094093cd6269 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.076262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88940998-499f-493c-8dae-55cc77f25593 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.114190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.114822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.115101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.115371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.120935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.121570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.121845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.122122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.128188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.132979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.133980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:56.134592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:56.140109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:56.152027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:56.153115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 356/690] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:56.153833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:56.160491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:56.163225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:56.164245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.164408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 335/691] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:56.164408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.300550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.331213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e31948-79b8-4faf-8ca6-dcbaa3ecbff7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.333846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b389928-49b0-45f0-8315-86e56c2fa6f3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.382151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.382765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.383039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.383310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.383977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:56.384307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.386438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.400226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.400840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.401116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.401388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.402082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:56.402393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.407824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.409749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 357/692] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:56.415822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.430511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 336/693] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:56.436602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.484179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.562266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cb64-4ef7-44cb-80cc-6fc0abc79249 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.569645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10198c1a-c2c6-4ff1-a0bf-5a1c49162f5e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.604868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.606257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.606762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.607297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.608326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=a8798813bd1e461d81faf73696f62a16) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:56.608856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.611501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.618406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 337/694] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/role_assignments?user.id=a8798813bd1e461d81faf73696f62a16&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:56.646301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.646975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.647328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.647606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.668621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.670358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-418a3557-5939-40fc-af44-52d84957eff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:56.695024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:56.696048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:56.697656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:56.697656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:56.697656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:56.697656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 358/695] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:56.702150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.716533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.726355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1192c1-8ffe-4ce1-92ef-e772f5cdd601 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.726666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1192c1-8ffe-4ce1-92ef-e772f5cdd601 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.726910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1192c1-8ffe-4ce1-92ef-e772f5cdd601 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.728681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 359/696] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:56.738616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.738616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.738616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.777844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:56.779048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:56.827108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7a3aa-723c-4eb3-8e67-5b2b0868e7e1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:56.867607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:56.868281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:56.868563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:56.868837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:56.869590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:56.869904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:56.871441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:56.883071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 360/697] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:47:56.886073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 338/698] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:56.895431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:56.942692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:56.998324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcce7c-4bb7-4bbb-ba5e-bb0d0b8a0b64 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.033600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.034237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.034515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.034786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.042329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.043771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:47:57.057616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:57.058633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:57.059572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 361/699] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:47:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:57.064883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.137382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:57.180064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.201474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819949fb-4a86-4cc6-8a77-9283fd504eb1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.277207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.277833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.278116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.278387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.279247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:57.279658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.281134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.319960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 339/700] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:57.327980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8508b-c19f-4753-a8b8-9cdf0e75ec95 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.330918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:57.336392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.364987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.365730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.366012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.366286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.366873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.368252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:57.380806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:57.381871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:57.382737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 362/701] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:57.387499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.522844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1feb62-0c4a-4bc4-a235-68e918177f3a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.527710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237c32-30d4-4190-853e-273bc4e6f4d1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.558635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:57.560627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.561523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.562041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.562746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.563582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:57.564100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.565742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc3611-cb6e-4925-b40d-000b4fc8a8fe None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.571048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.571307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 363/702] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:47:57.571557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.571944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.572253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.573509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5fb3-f3c3-4e47-8506-1d6e1e6e56ba None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.574052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5fb3-f3c3-4e47-8506-1d6e1e6e56ba None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.574531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5fb3-f3c3-4e47-8506-1d6e1e6e56ba None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.580896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 364/703] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:57.582013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b5fb3-f3c3-4e47-8506-1d6e1e6e56ba None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.616755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:57.659112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.660833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.695512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b5fb3-f3c3-4e47-8506-1d6e1e6e56ba None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:57.733369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:57.734387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.735454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.735922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.736806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:57.737320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:57.738871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0d705-a59d-4bb7-858c-393ee33e0f53 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:57.745519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 365/704] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:57.746921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:57.747513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:57.747989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:57.775912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:57.777469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:57.789429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 340/705] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 25 02:47:57.790409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:57.848729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:57.922661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 366/706] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:57.967168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e66a-b519-4074-93bf-2cfe86709591 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.003528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.004183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.004467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.004740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.015486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.017555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:58.034275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:58.035274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:58.036220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:58.036220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:58.036220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:58.036220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 341/707] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:58.041341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.070492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.148770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404963c-ba89-4b76-891a-c33762ce233b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.190379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.190920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.191201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.191476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.192130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:58.192506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.193987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.210732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 367/708] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:58.216157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.235454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.315438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.343271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821abb-a4b7-4097-94d1-8e3fe8687efd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.345584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.363943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46723e01-dcd6-4436-a374-763ddfb2694c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.386327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.386972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.387734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.388016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.396917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.398361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.412206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.412206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.412633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.412789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0469a686-9d7d-4eb2-8d18-ee6d31c19677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:47:58.418322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:58.419421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:47:58.420839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:58.428117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.428117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.428653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 342/709] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:58.430138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef46a-66d8-4ffe-87c5-17c074333cd1 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.430575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef46a-66d8-4ffe-87c5-17c074333cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.431019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef46a-66d8-4ffe-87c5-17c074333cd1 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.432680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 343/710] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:58.433382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fef46a-66d8-4ffe-87c5-17c074333cd1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6e037f00-c000-4344-ac69-e07bf40a86f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:47:58.445073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:58.446051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:47:58.446974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:47:58.446974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:47:58.446974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:47:58.449232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 368/711] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:47:58.450813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.451314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.451788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.473197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:58.474882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:58.553137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fef46a-66d8-4ffe-87c5-17c074333cd1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.563741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 369/712] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:58.564969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.602440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.603121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.603403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.603676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.604415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:58.604751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.606273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.628172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.628172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 344/713] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:58.635185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.664230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.727157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adced1-42bd-4376-b3ba-be7da3b87c7e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.795329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.795941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.796245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.796674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.797342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:58.797642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.798991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.812419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 370/714] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:58.818241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.828985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410674-33c8-4e3e-9d18-62f7c544bd7e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.898488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:58.901830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:58.902259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:58.902676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:58.903568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:58.904279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:58.907930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:58.914619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 345/715] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:58.915373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:58.959776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c5d68-c854-4160-bfc8-2632a6ad7c81 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:58.989236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:58.999498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.002435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.003949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.005300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.006393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:59.010449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d9231-35aa-4fb2-bb53-892f989d1b29 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:59.029142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:59.080613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 371/716] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:47:58 2022] POST /identity/v3/users => generated 254 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:47:59.087296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd551-c8fc-4197-a2e0-e7147f9f4a6b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:59.132184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.133193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.133679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.134159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] PATH_INFO: `/v3/services/network` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.134941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:59.136799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:59.138240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c27b-6b4e-41d3-ae15-76a7a5a3c8b8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.138524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c27b-6b4e-41d3-ae15-76a7a5a3c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.138800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c27b-6b4e-41d3-ae15-76a7a5a3c8b8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.141294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 372/717] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:47:59.147118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.147432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.147670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:59.152305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:59.153383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:47:59.154233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:59.157328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 346/718] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:47:58 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:59.163876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:59.183083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:47:59.184426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:47:59.265336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 373/719] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:47:59.278075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34857c-e861-4ac2-84b8-ab8ad4c5223e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:59.312561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:59.317561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.318399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.324576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.324576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.324576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:59.324576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:59.324576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:59.326976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 347/720] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:47:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:59.333070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:59.440608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2897f96-7b73-4a05-a32f-6e96f83a0001 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:59.455308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:47:59.478648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.479392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.479744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.480134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.480861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] RBAC: Adding query filter params (type=network) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:59.481235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:59.482661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:59.487633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 374/721] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:47:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:59.587094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:59.705545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d48f-5406-4508-ab58-bf4f0c3191b7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:59.744287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.744977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.745267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.745547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.756944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:47:59.762293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:47:59.763242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:47:59.764101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 348/722] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:47:59.770001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:47:59.921404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25e02d-510d-4448-be86-ca3dafca9389 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:47:59.972438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:47:59.973063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:47:59.973315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:47:59.973584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:47:59.974243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:47:59.974545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:47:59.975882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6292cea-e4e7-4d13-9f60-98db3cd54846 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:47:59.985428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 375/723] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:47:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:47:59.991392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:00.043167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4555a3-8ac5-49c4-8f2c-359bfcba82df None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.043529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4555a3-8ac5-49c4-8f2c-359bfcba82df None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.043723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4555a3-8ac5-49c4-8f2c-359bfcba82df None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.045573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 376/724] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:00.051600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.051881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.052167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.073043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:00.074262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:00.080982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:00.157016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:00.164479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b79d-f04f-4b7d-8deb-5ae6b6650a09 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:00.203553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 377/725] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:48:00.223764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:00.224423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.224705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.224974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.242680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:00.244238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:00.268794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:00.269878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:00.271369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:00.271369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:00.271369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:00.271369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 349/726] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:00.275812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:00.431939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635922-6622-4968-a2a3-94cb312567c7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:00.478039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:00.478576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.478857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.479133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.479782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:00.480083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:00.481469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:00.502242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 378/727] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:00.507693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:00.557507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:00.608496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:00.712587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:00.715848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f18fc-03ae-489e-9164-da6b483fa708 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:00.738854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f55c-7e99-4d9b-9106-bde5734c6f36 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:00.776599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:00.777572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.777855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.778185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.778914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:00.780340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:00.810608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:00.811322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:00.811606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:00.811880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:00.821667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:00.823210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:00.836692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:00.837888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:00.838844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 350/728] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:00.849113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:00.892647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 379/729] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:00 2022] POST /identity/v3/services => generated 251 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:00.999657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199f776-6dc9-4692-b310-f8e959c88506 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:01.044075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:01.044735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.045274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.045555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.046311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:01.046624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:01.048158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:01.072321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 351/730] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:48:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:01.077886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:01.177500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:01.187863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9593c-1706-439a-acb5-36d56172160e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:01.223966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:01.224620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.224946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.225221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.300620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:01.302051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7844-92f2-4615-8a6a-369bb14cb9d7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:01.407949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:01.418534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 380/731] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 25 02:48:01.424884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:01.455884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:01.569467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255e12-4c6c-422e-af99-bc30d83e0ad2 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.569773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255e12-4c6c-422e-af99-bc30d83e0ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.569947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255e12-4c6c-422e-af99-bc30d83e0ad2 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.571409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 381/732] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:01.576770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.577406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.577900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.598239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:01.599875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:01.607173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47222c2-3268-4db8-b8f7-5dac179755c1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:01.689961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:01.690624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.690897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.691175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.694995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 382/733] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:48:01.718148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:01.720531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-2e111158-629c-49f8-bda0-1556c0774648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:01.744279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:01.745301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:01.746212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:01.746212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:01.746212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:01.746212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 352/734] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:01.751211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:01.789099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81594d9-d978-4c08-b110-b90904ca2516 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.789430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81594d9-d978-4c08-b110-b90904ca2516 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.789647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81594d9-d978-4c08-b110-b90904ca2516 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.791579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 353/735] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:01.797452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311df7b-dcde-44cf-b0cb-70be67f11ba2 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.797741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311df7b-dcde-44cf-b0cb-70be67f11ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.798011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311df7b-dcde-44cf-b0cb-70be67f11ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.818049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-8311df7b-dcde-44cf-b0cb-70be67f11ba2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:01.819227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8311df7b-dcde-44cf-b0cb-70be67f11ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:01.884410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:01.893315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 354/736] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 25 02:48:01.894758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3208-41e9-44b0-9f45-ca9366713ddf None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.895040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3208-41e9-44b0-9f45-ca9366713ddf None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.895309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3208-41e9-44b0-9f45-ca9366713ddf None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.896747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 355/737] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:01.903082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:01.903369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:01.910426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:01.922232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5c3e7-25de-41f6-ac20-c36837d11366 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:01.925653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:01.927014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:01.979196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:02.007998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 356/738] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 25 02:48:02.032348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.032974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.033250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.033516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.033965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.034165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:02.034453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.035814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.061241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 383/739] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:02.066803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.178729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821223-4913-4fdd-8d45-b1002246a9a0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.216168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.217492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.218014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.218486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.228774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.233034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:02.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:02.250542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:02.251469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:02.253528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 384/740] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:02.258852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.266461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ded8-f068-4483-8a97-d42038ff5b62 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.294256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:02.304203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.304961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.305215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:02.305329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.305669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.316002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:02.320905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:02.322049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:02.323086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:02.328881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 357/741] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:02.330337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.372673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22905837-ee7f-40a9-9b73-5711c182cfab None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.411022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.411649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.411932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.412261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.412931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.414346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.508343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 385/742] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:02 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:02.509467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.583038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8034bc-016f-4579-bb57-c1147da8393b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.629211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.630210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.630681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.631136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.632101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:02.632650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.634310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.643408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:02.647581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14294d2-3b6d-496b-8127-32e0904eb507 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.649687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 358/743] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:02.650838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.700999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.701667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.701994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.702255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.707719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:02.718500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:02.724624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:02.725569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:02.726423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 386/744] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:02.727132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.784011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2b9d-552e-4146-86e3-ce25c40271c6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.820347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.821076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.821349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.821624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.822298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:02.822609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.824087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.834412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 359/745] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:02.835352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.837006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263cecd-c632-4a56-8c3e-a31e9a02fc31 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:02.874183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:02.874789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:02.875679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:02.875948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:02.876705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=a8798813bd1e461d81faf73696f62a16) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:02.877007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:02.878562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:02.884257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 387/746] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/role_assignments?user.id=a8798813bd1e461d81faf73696f62a16&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:02.884932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:02.946439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.020970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f431cec-9f13-4696-adac-dd6b45ec8a22 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.026955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.087840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f643-6495-4fa9-b111-fedc8bce3b01 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.098100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.098735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.099023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.099297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.099975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:03.100305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.101680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.119655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 360/747] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:03.126079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.126572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.127443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.128005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.128485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.154886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.158343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:03.183509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:03.184978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:03.185966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:03.185966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:03.185966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:03.189100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 388/748] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:03.195069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.286717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d020a4b-8a01-41b3-b49c-38170f7676b0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.327504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.328460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.328851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.329475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.342335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.344365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:03.362144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:03.363380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:03.364517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:03.364517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:03.364517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:03.365932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 361/749] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:03.371212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.393268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.402967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.431981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2805-fcb7-4cb9-9663-ebc0bf75e76e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.478333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.478963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.479239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.479509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.480164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:03.480579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.482092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8ec3-1382-431a-884c-5785edc8f0ea None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.495103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 389/750] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:03.496484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed3dd-276c-4670-86db-367602ced544 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.496767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed3dd-276c-4670-86db-367602ced544 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.497037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed3dd-276c-4670-86db-367602ced544 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.498563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 390/751] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:03.500439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed3dd-276c-4670-86db-367602ced544 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.527746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1722101-ea64-4412-b32c-3e8f232d7c23 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.580975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.581608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.582268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.582550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.583208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:03.583749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.586086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e75a86-c368-4ad3-b544-5df35a31323c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.602554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 362/752] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:03.604336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.604336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.604510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.627230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed3dd-276c-4670-86db-367602ced544 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.629528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:03.634237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:03.665483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.666353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.666894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.667239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.671184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.676247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.678572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:03.690967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:03.696779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 391/753] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:03.697864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.727557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 363/754] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:03.728916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.744317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:03.807984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbad1-2cf3-44fd-b925-7ba6c6450039 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.866104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921a6d2-0885-4326-bf3c-39de2e6188aa None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:03.872859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.873869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.874573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.875113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.892205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.894353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.905682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:03.906729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:03.907011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:03.907539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:03.913484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:03.913484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:03.913484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:03.930140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 364/755] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:03.933837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:03.934890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:03.935857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:03.936917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:03.942887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 392/756] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:03.943995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.043181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.048700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.109605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37866f-20f2-408e-9f5c-a70e9f30bd4d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.134158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b3f7-975c-4bad-9759-0bb1d89cd429 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.157733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.158413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.158648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.158923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.170433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.172370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.181059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.182250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.182935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.183435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.184451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:04.185359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.187112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:04.188089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:04.189317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:04.190430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:04.190430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:04.190430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:04.191147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 365/757] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:04.192016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.212127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 393/758] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:04.213177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.325913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300e4b-82a8-4d26-b9da-ed68a73aab6d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.333805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79ea41-ce8c-4407-a4d4-184972fa1198 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.352583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.353517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.363931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.365205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.365598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.366122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.366896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.368962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.376565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.377393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.377814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.378228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.379006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:04.379450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:04.381253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:04.382470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:04.383510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:04.384722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.391085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 366/759] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:04.391085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.403691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 394/760] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:04.404960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.518872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4894-424b-45a6-afc8-4af7909a81c4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.527679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451900a-7ac5-47cf-82b9-1a453266fe66 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.556407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.557015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.557292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.557567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.558193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:04.558493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.559842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.569623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 395/761] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:04.570474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.572003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.572879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.573395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.573815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.612634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.631977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.652338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.654027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.677447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62224a37-55d0-4480-bd24-e3c87eafb49c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.714548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.715124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.715403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.715671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.723821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.725394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:04.738255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:04.739372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:04.740316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 396/762] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:04.740808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.776309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 367/763] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:04.777168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.852306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.856172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e5723-d8cd-453a-b8e9-97a460adb63e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.867176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:04.885052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7f41-ffa8-4dfd-9a39-e0f3245dcf57 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:04.896846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.899266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.899266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.899266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.899266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:04.899707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.901360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab5003-e022-4783-a29a-b656296358d3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.913274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 397/764] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:04.913274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1824ea-9006-408a-a8cc-e2e659a7ab2e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.913274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1824ea-9006-408a-a8cc-e2e659a7ab2e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.913274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1824ea-9006-408a-a8cc-e2e659a7ab2e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.916818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 398/765] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:04.917485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1824ea-9006-408a-a8cc-e2e659a7ab2e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:04.928198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:04.928198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:04.928478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:04.929607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:04.931914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:04.932274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:04.933743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:04.946519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 368/766] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:04.947361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.030683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1824ea-9006-408a-a8cc-e2e659a7ab2e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.055136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63746881-66ab-4063-8c44-78549933afb3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.065244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.065870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.066150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.066423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.076619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.078573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.087869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.088512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.088796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.089072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.089770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] RBAC: Adding query filter params (interface=public, service_id=2759dc2e33ae49ad87a12f8288d13881, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:05.090082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.091439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-a2e8ff11-6dfe-44b7-a9be-2618f750bec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:05.092457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:05.093438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 399/767] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3df36-d89b-46e9-950c-9bfec411a3d2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.095699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3df36-d89b-46e9-950c-9bfec411a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.096045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3df36-d89b-46e9-950c-9bfec411a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.103575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 369/768] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:04 2022] GET /identity/v3/endpoints?service_id=2759dc2e33ae49ad87a12f8288d13881&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:05.103575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.109326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.115078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-08d3df36-d89b-46e9-950c-9bfec411a3d2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:05.116323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-08d3df36-d89b-46e9-950c-9bfec411a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:05.126470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.183942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 400/769] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:05.185435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc2c5-fac0-4f40-9547-59b3c0610b59 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.185985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc2c5-fac0-4f40-9547-59b3c0610b59 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.185985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc2c5-fac0-4f40-9547-59b3c0610b59 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.187688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 401/770] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:05.188412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dc2c5-fac0-4f40-9547-59b3c0610b59 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.215746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefa91a-fbe6-4f82-87e5-81c498b74412 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.253301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72031492-a392-469a-b567-403fcf80686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.253962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031492-a392-469a-b567-403fcf80686d None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.254247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031492-a392-469a-b567-403fcf80686d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.254523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72031492-a392-469a-b567-403fcf80686d None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.256485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72031492-a392-469a-b567-403fcf80686d None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=867d21f5f80d4cc48cc53e0b89c7f2f8, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.257932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72031492-a392-469a-b567-403fcf80686d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.299734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dc2c5-fac0-4f40-9547-59b3c0610b59 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.331831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 370/771] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:05 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:05.332886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72031492-a392-469a-b567-403fcf80686d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.343354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.344247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.344598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.350862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.350862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:05.350862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.350862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0aed60-bdc9-45f7-83ed-fdf156bb1545 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.367266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 402/772] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:05.367266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.367266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.367266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.374487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.387540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:05.389168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:05.428597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.470182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72031492-a392-469a-b567-403fcf80686d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.483118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 403/773] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:05.483118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.511813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.512457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.512739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.513065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.513672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.515810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.521084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 371/774] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:05.521845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.645833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7eaa5a-b1ec-43bf-b2e5-d610f47095cc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.711790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41ad1e-7a56-49d3-baea-0d3d3c736131 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.728785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.729762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.730044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.730311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.732805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.741163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.744467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.753558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:05.759372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.760010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.760997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.760997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.761219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:05.761710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:05.762721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:05.763804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:05.764541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.767607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 404/775] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:05.773187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:05.773463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:05.774447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:05.775290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 372/776] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:05.780652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.897471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f911a5-ad2c-435b-ae91-15306ff4b4d5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.906508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175920e-2fb1-4115-b339-a12430ff82a7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:05.950845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.951473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.951753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.952025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.953643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:05.953958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.955129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:05.955342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.956138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:05.956642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:05.957105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:05.958000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:05.958516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:05.960126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:05.960517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 373/777] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:05.961318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:05.973825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 405/778] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:05.975030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.022252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.036176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.068909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e30de-4736-4a71-82dd-0f2915812bb0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.083605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415f94-1060-4d95-b170-e2c176b42c23 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.105095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.105731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.106014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.106288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.118358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:06.121082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.121955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.122450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.122922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.123754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:06.124286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:06.124612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:06.126400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 374/779] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:06.127750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.129986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.137130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 406/780] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:06.138176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.240582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e64b-e9e2-429d-9e5b-d2e56acc3548 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.247351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15dbb6-f8b5-491f-892c-40c4a1c2c6d3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.258340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.267298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.276906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.277455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.277740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.278184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.278729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=66f04604a2f44191a8af25bab59e44bc) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:06.279038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.280499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.286339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 375/781] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/role_assignments?user.id=66f04604a2f44191a8af25bab59e44bc&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:06.287198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.287461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.288356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.288840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.289326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.290211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:06.290743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.292405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.304283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 407/782] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:06.309825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.428467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf870dc6-087e-414c-bb12-e35c50533500 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.468954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.470460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.470959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.471433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.483982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.486459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:06.505823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:06.506832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:06.507734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:06.507734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:06.507734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:06.510327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 408/783] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:06.515868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.560826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ff3a-40df-482d-b681-5f1a68c3dcb3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.563860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.630843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.631758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.635257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.635997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] RBAC: Adding query filter params (interface=public, service_id=f680e2087df240218b20bf8edb2792ab, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:06.636344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.637741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.646672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 376/784] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/endpoints?service_id=f680e2087df240218b20bf8edb2792ab&interface=public®ion_id=RegionOne => generated 194 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:06.652181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.681288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e520355-d32d-4bfd-b779-f9d8c1eed377 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.717859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.718997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.719280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.719549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.720221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:06.720534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.721908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.734744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 409/785] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:06.740942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:06.741534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.804645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf4f56-8b34-47f0-8de3-edb0af7b51d6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.841654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:06.853583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63472d57-12ff-4f85-8876-cdcbe5af5627 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:06.878199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.878819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.879102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.879374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.880019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.881391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.892209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:06.892915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:06.893200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:06.893475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:06.896335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 377/786] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/services => generated 772 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:06.902027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:06.903534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:06.919515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:06.920771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:06.921929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 410/787] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:06.928178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:07.074440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1c920-c29b-44bd-a430-699804c3b288 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:07.088515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:07.125820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:07.126604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.126893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.127170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.127923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:07.128257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:07.129716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:07.151747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 378/788] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:48:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:07.157626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:07.284321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02a81b-c233-4002-88fc-d6b1b858481a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:07.319860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:07.320524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.320802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.321073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.332378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:07.333843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:07.348929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:07.350972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:07.353102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:07.353102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:07.353102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:07.353102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 411/789] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:07.355913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:07.479699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5455c-3fe2-4a67-b6b8-6e7ef61a4be1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:07.525772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:07.526760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.527248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.527850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.528753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:07.529263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:07.531354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0791cc-68f8-4cdf-9b1b-3d87740dc049 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:07.548983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 379/790] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:07.554262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:07.558409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:07.623383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96ae2-fa4e-4db3-bbf3-618a05aba3f9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.624026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96ae2-fa4e-4db3-bbf3-618a05aba3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.624928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96ae2-fa4e-4db3-bbf3-618a05aba3f9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.626842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 380/791] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:07.632562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef19a7a-8578-4518-8e36-ef2563c13486 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.633115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef19a7a-8578-4518-8e36-ef2563c13486 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.633791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef19a7a-8578-4518-8e36-ef2563c13486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.724824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-eef19a7a-8578-4518-8e36-ef2563c13486 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:07.726508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eef19a7a-8578-4518-8e36-ef2563c13486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:07.744691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2ab9-ba77-42a7-8e9d-311d9413a008 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:07.758557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:07.789921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:07.790894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.791177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.791451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.800505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:07.802460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:07.809633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 381/792] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:07.815221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:07.816126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:07.817325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:07.819547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9758-ed62-44a6-bccd-5c562048655e None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.820047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9758-ed62-44a6-bccd-5c562048655e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.820568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9758-ed62-44a6-bccd-5c562048655e None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.820930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 412/793] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:07.822179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:07.822755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 382/794] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:07.827692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.828783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.829343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.861142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:07.862681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:07.898351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:07.943734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa000a-5a2d-4b6c-8856-7151f0b304ea None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:07.982648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:07.983320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:07.983602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:07.983874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:07.985141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:07.985685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:07.987218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.007594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 413/795] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:08.008909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.024001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 383/796] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:08.121546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea87fa-9df4-476f-a489-db8f18a00572 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.146461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:08.158544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.159214 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.162388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.162388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/a8798813bd1e461d81faf73696f62a16/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.162388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=a8798813bd1e461d81faf73696f62a16, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:08.163065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.196534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.240674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 414/797] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:08 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/a8798813bd1e461d81faf73696f62a16/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:08.322207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:08.337508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4fc44-7078-4d2a-a81c-c1be4b68f3ce None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.368907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.388607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.396599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.396947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.397237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.410220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:08.418855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:08.420086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:08.421142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:08.424178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 384/798] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:08.430212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.485722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3940ce9-8e60-4af8-85d2-747f6f198583 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.522722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.523469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.523900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.524348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.525020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:08.526620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:08.537341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:08.538428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:08.539624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 415/799] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:08.545126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.598388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:08.652638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c1ed-5256-4778-b329-0f46415b6097 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.690176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.690790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.691116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.691396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.692032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:08.692370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:08.693855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.698308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8bc69-4fa6-48da-86df-35c3431b451d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.704070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 416/800] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:08.705329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.794988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.795975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.796527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.797251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.798260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:08.798774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:08.800395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.814905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aab96-786f-4758-a22f-b074d5afa0d5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:08.818725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 385/801] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:08.824407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.845048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:08.848228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:08.853914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:08.854519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.854802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.855123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.855770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:08.856565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:08.857578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9d861-f02a-4712-a19c-6d29cb57e40b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:08.863027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 417/802] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:08.864046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc90d-8cb2-47f8-aef9-ceba3b838ea0 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:08.868466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc90d-8cb2-47f8-aef9-ceba3b838ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:08.868824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdc90d-8cb2-47f8-aef9-ceba3b838ea0 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:08.870384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 418/803] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:08.871934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdc90d-8cb2-47f8-aef9-ceba3b838ea0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:08.972340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55a6c-f6ff-4295-a8d3-dae091b157b8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.008077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdc90d-8cb2-47f8-aef9-ceba3b838ea0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.017871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.018520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.018802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.019079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.036136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:09.037958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:09.047962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.048863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.049363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.049838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.050689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:09.052305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e8e925f2-da29-4aad-90a7-ec7a19ecce7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:09.055140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:09.056054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:09.056996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:09.056996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:09.056996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:09.060229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 386/804] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:09.061832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f0171-acb8-40a5-808e-42744100845b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.062691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f0171-acb8-40a5-808e-42744100845b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.063120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f0171-acb8-40a5-808e-42744100845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.084175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f28f0171-acb8-40a5-808e-42744100845b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:09.085392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f28f0171-acb8-40a5-808e-42744100845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:09.151490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 419/805] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:09.151490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.169386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.222099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 387/806] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:09.224539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.289486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb827b-b608-4c77-8c40-0053daa751d8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.331262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.332132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.332589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.333010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.333917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:09.334400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:09.336043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275e6ae-36d0-40be-b334-99771d9964ac None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:09.352490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 420/807] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:09.354279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f0171-acb8-40a5-808e-42744100845b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.397463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42cfe-1b2f-4edb-9dd1-f412cd627b1c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.398001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42cfe-1b2f-4edb-9dd1-f412cd627b1c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.398001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42cfe-1b2f-4edb-9dd1-f412cd627b1c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.399339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 421/808] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:09.404592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.404809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.405108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.425920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:09.426631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:09.483968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f0171-acb8-40a5-808e-42744100845b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.511284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 422/809] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:09.529805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.530452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.530738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.531621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.540049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:09.541668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:09.555630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:09.556669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:09.557634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 388/810] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:09.563026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.567156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.598072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.644337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.732797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eeef8-1629-4b26-a261-5335994d6793 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.746755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc96b4-a0c8-450f-b1e3-375f101cd37a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.793200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.793843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.794122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.794397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.806334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:09.811640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:09.813280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:09.814123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:09.818604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 389/811] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:09.819801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.831766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.832434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.832711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.832987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.833730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:09.834037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:09.835652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:09.860883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 423/812] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:09.861853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:09.952710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09eb4-2fa7-40ee-adc4-ab12baf477d3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:09.964367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.975660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:09.994530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:09.995629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:09.996120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:09.996697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:09.997721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:10.005131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094b610-fda2-4977-8ee0-7d07c0089842 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.005257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.005257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.010056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 390/813] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:10.011163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.050931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.051562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.051842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.052190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] PATH_INFO: `/v3/services/network` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.052941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.055106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:10.067454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:10.068563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.069427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 424/814] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:09 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:10.070223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.135279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf753a9-eef3-4f9d-8ffe-0f0b98e4af8c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.172920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.174072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.174656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.175136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.196836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bd39f-c311-42ae-a551-1ff9c558709f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.220477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.239072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.239843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.240130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.240432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.255440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.255908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.256869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.258084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.259474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:10.275279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:10.276353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:10.277333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:10.277333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:10.277333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.280229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 425/815] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:10.281380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.385652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 391/816] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:10.385652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.424413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68906163-1f9d-46b7-b57f-b6de6eb08e05 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.460414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.461028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.461307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.461586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.462211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] RBAC: Adding query filter params (name=network) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:10.462519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.463870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.468985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.469305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 426/817] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:10.470442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.506889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ea97b-9b75-4337-ae7f-5564d797ec8e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.536508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.544219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.545077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.545356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.545778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.547048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:10.547373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.548759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5101576-7661-40d5-8a07-f3a5bea26da7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.562579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 392/818] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:10.563944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6091-4adc-4e3f-9a77-75971a8788e0 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.564255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6091-4adc-4e3f-9a77-75971a8788e0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.564526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6091-4adc-4e3f-9a77-75971a8788e0 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.570043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 393/819] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:10.570043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6091-4adc-4e3f-9a77-75971a8788e0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.595269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76a53b-68d9-4e52-bf8b-37d790f6b607 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.633353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.634008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.634490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.634600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.645801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.647231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:10.664426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:10.670653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:10.672317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:10.672317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:10.672317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.672317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 427/820] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:10.672317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.685357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6091-4adc-4e3f-9a77-75971a8788e0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.716392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.722952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.723625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.730687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.730687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.730687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] RBAC: Adding query filter params (type=network) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:10.730687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.730687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1232ad-5d9a-4998-a6ef-ee327d93816f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.732096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 394/821] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:10.733335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.733622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.733949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.752751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:10.753948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:10.780343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964800fb-50bd-4d47-8d44-82592cb3f7b9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.794253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.817167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.817957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.818570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.818904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.822559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 395/822] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:10.828190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.829695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:10.833036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:10.842434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:10.843945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:10.844910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:10.849166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 428/823] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:10.850002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:10.953451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e5885-9b2e-4797-acfc-5c54419e6a06 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.982556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:10.986220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd53f9-e041-4893-a6a4-b7565277c6d9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:10.994535 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:10.995298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:10.999582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:10.999582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:10.999582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:10.999582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:10.999582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.016182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 396/824] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.016182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.037096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.038347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.038829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.039330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.040236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.042565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.115385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 429/825] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:11.115385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.136960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f162342-42a3-49e6-b05e-7f5508c1f879 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.145957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.195718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.196030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.196459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.196810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.197658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:11.198165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.199679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.226859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 397/826] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.226859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8212-cf26-46d7-9a85-16a6865f7595 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.275564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.310195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.310830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.311107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.311430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.320367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.321892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:11.341019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:11.341981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:11.342868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:11.345677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 430/827] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:11.346668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.359389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37234efb-74c0-4e67-a220-53690dadd7a2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.398537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.399177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.399458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.399733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.414983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:11.419991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:11.421058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 398/828] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:11.425112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.450130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.462578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f152d13-6560-43ca-833d-d06a215c8ddb None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.495191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.495816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.496096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.496401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.507364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.508839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.518806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.522562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbe20-deb6-4d54-af46-1be6d399ec42 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-72b19eb3-0cb2-448f-afe8-6075b8436c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:11.523503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:11.524650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:11.525762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:11.525762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:11.525762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:11.525762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 431/829] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:11.525762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cdfb8-8201-4cf0-97e7-c53e9c052f06 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.526116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cdfb8-8201-4cf0-97e7-c53e9c052f06 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.526284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cdfb8-8201-4cf0-97e7-c53e9c052f06 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.527490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 432/830] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:11.528187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cdfb8-8201-4cf0-97e7-c53e9c052f06 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.564175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.564720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.565002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.565286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.565984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:11.566300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.567752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.584438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 399/831] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.584438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.631537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cdfb8-8201-4cf0-97e7-c53e9c052f06 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.664439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.665044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.665326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.665602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.666260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:11.666574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.667929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298805-645d-45bd-9b4e-4d186fd9ceaf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.677419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 433/832] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.678607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.678890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.679160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.687916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.690341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905038b-c08f-4300-95a1-579cd5635c70 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.699252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:11.700473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:11.723378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.723978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.724288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.724560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.725194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:11.725498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.726853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.738838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 400/833] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.739616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.758266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.770565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 434/834] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:11.771394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.860015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee89c-02e9-42dd-a65a-28351a81a19b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.875370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d2d11-c9ba-4859-af39-045e63253840 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:11.899469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.900412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.900932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.901357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.902251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:11.902826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.903631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:11.905283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.909223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:11.909854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:11.910126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:11.910398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:11.920517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 401/835] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:11.927785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:11.928020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:11.928020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:11.948705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:11.949754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:11.950895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:11.950895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:11.950895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:11.950895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 435/836] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:11.958077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.035713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.058618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba245ef1-a186-4ec8-89b3-e612f2c598f0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.102532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.103332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.103617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.103889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.111282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.112721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:12.124271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:12.126186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:12.127098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:12.128308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 402/837] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:12.129126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.209217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b787d9c-0cb6-41dc-9a60-96b9220f2779 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.241609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aba17-e42f-4e2c-b24f-de580b003edb None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.246486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.247354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.247840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.248346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.249310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:12.250364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.252112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.269606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 436/838] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:12.270827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.276227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.276870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.277154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.277430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.278026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.279441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:12.288269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:12.289490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:12.290346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 403/839] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:12.291155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.301437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.333248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.392415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9631d-f69f-4199-8aaf-b562894713ed None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.399726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d577d-53ea-4dc0-9e18-4447eee2fe2b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.426544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.427152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.427431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.427697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.428552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:12.430349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.430583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.439702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.440496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.440775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.441491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.449152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 437/840] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:12.449152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.454414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.456311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:12.471833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:12.472936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:12.473887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 404/841] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:12.475146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.548029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.551584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115d3a-1b03-4981-b531-3a3ce9855406 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.555240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.583576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979841c9-6cc1-4d50-8988-e2dca5d74603 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.588797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.589407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.589683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.589950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.590580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:12.591309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.593314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.599022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 438/842] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:12.600077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.619879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.620621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.620925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.621177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/66f04604a2f44191a8af25bab59e44bc/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.637163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=66f04604a2f44191a8af25bab59e44bc, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.638825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.706491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c15f9-954e-491e-a651-1875e95eba7a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.740359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.740994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.741274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.741542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.742260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:12.742566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.743966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.765214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 405/843] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:12 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/66f04604a2f44191a8af25bab59e44bc/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:12.766880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.773512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 439/844] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:12.774732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:12.819987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.888811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f258f-8b23-44e6-89d4-f3578fcf6bdd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.911833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:12.926903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed247-81c9-47aa-9570-70b8b8b0c8d7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:12.948301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.948851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.949129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.949399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:12.950507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:12.950965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:12.953021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:12.959086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 406/845] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:12.969350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:12.969981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:12.970262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:12.970533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.051270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:13.053154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:13.085661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:13.169785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 440/846] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:13.175244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:13.290448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dbfec-0642-44dc-aeb9-d0cb27d8eb7f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:13.327391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:13.328901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.329181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.329455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.340678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:13.342138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:13.357253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:13.358208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:13.359730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:13.359730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:13.359730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:13.359730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 407/847] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:13.366019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:13.479670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb0e2-a9a7-4a29-98f5-b5c967c9bf67 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:13.507202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:13.518526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:13.519386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.525107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.525107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.525107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:13.525107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:13.525107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:13.540523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 441/848] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:13.542214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:13.589000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:13.648713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804a78e-70ce-4674-be2e-c5dd49cd590d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:13.685137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:13.685770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.686045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.686313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.693388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:13.694858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:13.706461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:13.707372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:13.708321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 408/849] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:13.713523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:13.820101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687e57c-1836-4516-a0b3-ff5df0f63fe5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:13.855621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:13.856258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.856585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.856858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.857555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:13.857858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:13.860052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa57a-42df-4ff9-8f8e-fac75c80ece9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:13.873177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 442/850] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:13.880035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:13.934813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77854898-dd45-4b5b-894f-ac2b926d7762 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.935092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77854898-dd45-4b5b-894f-ac2b926d7762 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.935361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77854898-dd45-4b5b-894f-ac2b926d7762 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.937696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 443/851] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:13.945298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:13.945650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:13.945988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:13.971274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:13.973280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:13.989987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:13.999453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba2152-36c0-434a-8f76-f14d03e50e7f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:14.036696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:14.039342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:14.039957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.040269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.040615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.041413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=a8798813bd1e461d81faf73696f62a16) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:14.041727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:14.043222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:14.056176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 409/852] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:13 2022] GET /identity/v3/role_assignments?user.id=a8798813bd1e461d81faf73696f62a16&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:14.072318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 444/853] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:14.344173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:14.449398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034af9-8115-4c90-812d-00f5af59acfc None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:14.486883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:14.487801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.488305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.488721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.489450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:14.490982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:14.500899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:14.502059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:14.503167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:14.505552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 410/854] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Aug 25 02:48:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:14.513522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:14.620629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff514472-8084-4020-a0ce-e39f459c9a8f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:14.657273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:14.658125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.658547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.658996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.659767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:14.660233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:14.661721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba3a0e-5bb2-434b-8aab-85d9cad93024 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:14.666898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 445/855] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:14.673256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:14.696296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:14.779370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9e860-33af-41e9-9ddb-b143ea64acc9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:14.816204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:14.817049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.817499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.817913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.818687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:14.819132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:14.820664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721d40a-2f28-431b-84c7-2dc71f86c571 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:14.825661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 411/856] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:14.869075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef320-7ca2-4821-80f6-346671aaaee5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.869611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef320-7ca2-4821-80f6-346671aaaee5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.869611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef320-7ca2-4821-80f6-346671aaaee5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.871348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 446/857] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:14.878324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18622-1be9-4f41-8a7d-1ec02caef64a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:14.878610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18622-1be9-4f41-8a7d-1ec02caef64a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:14.878886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa18622-1be9-4f41-8a7d-1ec02caef64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:14.899000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1fa18622-1be9-4f41-8a7d-1ec02caef64a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:14.900251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1fa18622-1be9-4f41-8a7d-1ec02caef64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:14.989685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 412/858] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:15.017740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192f45b-0921-4275-8cbf-556c051d45a1 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.018028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192f45b-0921-4275-8cbf-556c051d45a1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.018302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192f45b-0921-4275-8cbf-556c051d45a1 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.019927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 447/859] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:15.025354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.025648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.025928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.045184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:15.047128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:15.057042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:15.127356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 413/860] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:15.270046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192f45b-0921-4275-8cbf-556c051d45a1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:15.317752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:15.379690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192f45b-0921-4275-8cbf-556c051d45a1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:15.401933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:15.416942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:15.417597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.417877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.418150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.431677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:15.436446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:15.437583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:15.438453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 448/861] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:48:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:15.445513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:15.530182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699871b-91b0-4c44-ab28-facba2b60bd3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:15.576039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:15.579304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:15.579778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49f941-7b76-46a4-8ca2-ca60ea1b0209 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:15.580788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.581417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.581853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.584270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:15.585156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:15.616168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:15.616788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.617080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.617331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.618039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:15.618343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:15.619690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:15.638204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 449/862] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:15.638204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:15.664358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 414/863] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:15 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:15.755438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba0c23-6337-4a70-adc5-ef864dd98ddc None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:15.808109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:15.808776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:15.809053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:15.809322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:15.820945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:15.822468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:15.839625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:15.840663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:15.841635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:15.841635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:15.841635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:15.841635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 450/864] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:15.846955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:15.850783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:15.964611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18645d57-7f0c-43b7-b842-bc0fc02cc8aa None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.001125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0870796-d698-482f-bce6-483631f95bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.001736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0870796-d698-482f-bce6-483631f95bab None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.002019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0870796-d698-482f-bce6-483631f95bab None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.002296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0870796-d698-482f-bce6-483631f95bab None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.002947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0870796-d698-482f-bce6-483631f95bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:16.003257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0870796-d698-482f-bce6-483631f95bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.004670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0870796-d698-482f-bce6-483631f95bab None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.021915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 415/865] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:16.025256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.039297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.132695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0e66b-08a6-48ef-b2c3-f58814f7f014 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.167400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.168035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.168346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.168622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.177774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb75a5-8e1c-460a-9458-dd06fa4ce033 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.178059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb75a5-8e1c-460a-9458-dd06fa4ce033 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.178331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb75a5-8e1c-460a-9458-dd06fa4ce033 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.178547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.180038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.180169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 416/866] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:16.185191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.185406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.185676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:16.192818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:16.193792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:16.194767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:16.196236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 451/867] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:16.201145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.208807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:16.210117 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:16.273490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.274059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.291770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 417/868] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:16.306336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5885-c7ff-42cf-95da-65ee2b45f3af None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.342747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.347197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.369341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 452/869] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:16.371386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.478154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06c4db-a645-4bed-a873-c299d9e9caaf None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.512772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.513433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.513715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.513986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.566567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.579836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.587744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.589232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.700189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 418/870] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:16.705640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.716554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc381-a83e-46fe-9cae-98241bc6355a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.760591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.771035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.772094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.772421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.772845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.773564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.782567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:16.788578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:16.789574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:16.790397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:16.791131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 453/871] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:16.797992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.816016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:16.823532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdf3ae-c5f5-4b3c-bbf1-4aca4d411ae4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.870759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.871388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.871672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.871950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.882778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.884262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-edb9a7d0-ec9f-45a0-a8ca-09bfd750a7cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:16.900164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:16.901149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:16.902051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:16.902051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:16.902051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:16.906514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 419/872] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:16.908021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67716-e589-41b5-a78f-b52740c8d89a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.908330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67716-e589-41b5-a78f-b52740c8d89a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.908602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67716-e589-41b5-a78f-b52740c8d89a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.912185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 420/873] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:16.913469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67716-e589-41b5-a78f-b52740c8d89a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:16.921339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead70bbb-cfb4-49ce-940f-53c85b0aeb44 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:16.958970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:16.959149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.959346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.959606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.960259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:16.960566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:16.961908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53062f04-6da1-4d1c-ac37-0bfabd246d7c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:16.967116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 454/874] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:16.969720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:16.970282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:16.970784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:16.990682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:16.991863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:17.029905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67716-e589-41b5-a78f-b52740c8d89a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.043750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:17.065708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 455/875] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:17.066655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:17.078898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.080134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.080652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.081408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.082669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:17.083428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.085498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.102228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 421/876] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:17.107822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:17.120914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:17.197333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19a63f-aeb9-4676-8f9a-bfcb1489bc2c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.235822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.236817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.237303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.237827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.238656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:17.239161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.240588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27826ca7-96db-40b6-9bdc-08badd96310d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.240985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.251626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 456/877] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:17.282652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.283728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.284317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.284800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.293441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.295575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:17.309214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:17.310193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:17.311121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:17.314460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 422/878] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:17.322478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:17.340999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:17.391922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:17.426222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:17.458970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0c8f9-27c8-4d4c-ac04-424ea4f6af22 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.476070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5af4-fa2a-4de3-a553-9d0c352b4bd9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.503222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.504391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.504903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.505382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.506278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:17.506883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.508706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.520668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.521302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.521572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.521838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.522486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.523869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc9917-ed5e-49a9-8478-ff31bb1c6700 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.524496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 457/879] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:17.530415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:17.603174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 423/880] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:17 2022] POST /identity/v3/services => generated 269 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:17.655097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194016e-5703-4c16-bb2f-592db15fb8d0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:17.668364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:17.701293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:17.702036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.702819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.703130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.703864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=66f04604a2f44191a8af25bab59e44bc) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:17.704184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:17.705629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e73e-22f3-4ffa-b3ae-f8506c99b1fb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:17.712837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 458/881] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:17 2022] GET /identity/v3/role_assignments?user.id=66f04604a2f44191a8af25bab59e44bc&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:17.723029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1808a-bdb0-4386-a387-fa1404e558b2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.723363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1808a-bdb0-4386-a387-fa1404e558b2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.723576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1808a-bdb0-4386-a387-fa1404e558b2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.725392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 424/882] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:17.731400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:17.731739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:17.731959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:17.754544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:17.755769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:17.838098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 459/883] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:18.132245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1808a-bdb0-4386-a387-fa1404e558b2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:18.241289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1808a-bdb0-4386-a387-fa1404e558b2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:18.269593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:18.278916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:18.279549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:18.279826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:18.285930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:18.285930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:18.285930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:18.291501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:18.292534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:18.293796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 425/884] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:18.301630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:18.415985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:18.419861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696b667-355d-4ebc-ac3a-3837637a8b48 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:18.457086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:18.457709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:18.457992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:18.458265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:18.458903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:18.459209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:18.460605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:18.465658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 460/885] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:18.473327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:18.587004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443979f-67ab-4129-a419-ab29887d68b0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:18.622699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:18.623355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:18.623642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:18.623921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:18.625161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:18.625510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:18.626946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:18.632591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 426/886] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:18.641509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:18.752790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c59c7b-a29d-4d38-b328-aa46b83af0a8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:18.789396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:18.790306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:18.790761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:18.791206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:18.792077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] RBAC: Adding query filter params (interface=public, service_id=a1e0c7cb881140dc97d427e331b7aba3, region_id=RegionOne) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:18.792660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:18.794479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e307a-0b31-44eb-acde-a29e98080eea None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:18.803045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 461/887] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:18 2022] GET /identity/v3/endpoints?service_id=a1e0c7cb881140dc97d427e331b7aba3&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:18.808993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:18.870007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:18.924178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac74489-8b04-475e-b53f-954f5fea52d5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:18.962044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:18.962659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:18.962940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:18.963206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:18.963821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:18.965353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af590-a471-4bad-a4d9-4990f232423e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:18.972068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 427/888] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:18 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:48:19.262612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3346b-b344-40f7-a58d-c1cf55c780f6 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.262806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3346b-b344-40f7-a58d-c1cf55c780f6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.263075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3346b-b344-40f7-a58d-c1cf55c780f6 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.264680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 462/889] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:19.270628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e630ed-148b-4ceb-8fe9-21fc70132f79 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.270924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e630ed-148b-4ceb-8fe9-21fc70132f79 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.271200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e630ed-148b-4ceb-8fe9-21fc70132f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.289893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a0e630ed-148b-4ceb-8fe9-21fc70132f79 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:19.291126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a0e630ed-148b-4ceb-8fe9-21fc70132f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:19.366222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 428/890] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:19.641718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3346b-b344-40f7-a58d-c1cf55c780f6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:19.674006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:19.754356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3346b-b344-40f7-a58d-c1cf55c780f6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:19.787164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adeb21-fdc2-47e3-b1a6-8add2e6a50d4 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.787698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adeb21-fdc2-47e3-b1a6-8add2e6a50d4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.787788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adeb21-fdc2-47e3-b1a6-8add2e6a50d4 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.789363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 429/891] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:19.791702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:19.792336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.792618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.792894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.793811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:19.794527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.794874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.795212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.795430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c78062-3b1b-43b2-afea-ca21e759cb81 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:19.813766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:19.815210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:19.853148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 463/892] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:19 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:19.854427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3d9f-7947-4b19-be35-60431771fd8b None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.854748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3d9f-7947-4b19-be35-60431771fd8b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.855023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3d9f-7947-4b19-be35-60431771fd8b None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.856608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 464/893] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:19.862489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:19.862795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:19.863033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:19.878782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:19.884260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:19.885473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:19.921600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 430/894] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:19.976903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 465/895] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:20.213164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.261854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.281568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:20.344601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba7feb-ba86-4c83-9f6e-2780eeb3ff3b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.372583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67316bef-989b-42ba-a6ce-d3b9f84271df None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.394069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.395562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.396371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.396847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.397899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.400132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:20.413009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:20.414013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:20.414924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:20.415655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 431/896] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:20.422056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.422056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.422056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.422056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.423599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.434052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:20.438922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:20.439840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:20.440724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:20.444465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 466/897] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:20.446230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.470928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:20.531849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:20.538337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667d219-d34e-47f0-a1c0-c05708594bfe None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.555357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b24d0-d8c9-493a-bdb7-bdd9f1795f17 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.576036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.578268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.578752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.579245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.580081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:20.580681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.582347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.592391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 432/898] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:20.594352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.597539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.598113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.603815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.603815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.603815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:20.603815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.603815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.615056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 467/899] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:20.616176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.705404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe1d9c-1cd4-482e-a3bb-d057c6a2de42 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.707491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:20.722621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04869d-5f87-4cbc-beb3-40633d7cd6c7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.744861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.745790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.746419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.746914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.747858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:20.748508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.750900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.762436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 433/900] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:20.763184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.763814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.764089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.764404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.765759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.776488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:20.778839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.780376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:20.795384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:20.798411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:20.799381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:20.799381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:20.799381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:20.802255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 468/901] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:20.803406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.883513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204698ac-a646-4cac-917c-befaf6955f8f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.910856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b391-0b08-45c7-b653-eba0ce6d7d67 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:20.922819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.923660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.924123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.924565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.925539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] RBAC: Adding query filter params (interface=public, service_id=4ccca0c270c14187b9cefbc6b9292b9b, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:20.926061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.927787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.944717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 434/902] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/endpoints?service_id=4ccca0c270c14187b9cefbc6b9292b9b&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:20.944717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.948495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:20.949107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.949396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.949667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.950313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:20.950616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:20.951981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa74933-f542-4f3e-957d-6f3333e3d087 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:20.965087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 469/903] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:20.966404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12436e-983d-488c-a140-a0821a5c0e69 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:20.966689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12436e-983d-488c-a140-a0821a5c0e69 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:20.966956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12436e-983d-488c-a140-a0821a5c0e69 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:20.969798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 470/904] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:20.970623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12436e-983d-488c-a140-a0821a5c0e69 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:20.981630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.028752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.060193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab81e-e2d7-4d1f-8909-a94256cfecf7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.079676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12436e-983d-488c-a140-a0821a5c0e69 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.095070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.095683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.095964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.096263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.096870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.098241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a666070-b54e-48e6-945f-15a5170d1323 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.103963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 435/905] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:48:21.105153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.105438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.105713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.126822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.127731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.128210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.128638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.129868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:21.131474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:21.138250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.140493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:21.154124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:21.155509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:21.157067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:21.159296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 471/906] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:21.165503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.218215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 436/907] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:21.280302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874693-5118-4a19-92a6-0b2876511aec None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.295416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.318099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.318715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.318995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.319267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.319993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:21.320334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.321749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.344325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 472/908] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:21.346036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.373787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.457301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37053cb-3abb-48fd-a059-5a0cda31b2fc None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.495036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.498722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.499887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.500188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.500580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.585040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.587018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.610109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c4cf-02d2-4f5b-a0af-ec40ea8c4471 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.619524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.648054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.649796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.650194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.650576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.651270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.652988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:21.663344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:21.664492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:21.665354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 473/909] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:21.674056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.710448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 437/910] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:21.715369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.716184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.789330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17013067-5e3e-45d4-934a-3c69f0649c32 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.836222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.837805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.838566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.839030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.840231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:21.841124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.843713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.850495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602df3-3c99-4abc-97fc-66d4d4f0a7a8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:21.851105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 474/911] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:21.859330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.888523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.900387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:21.901313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.901820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.902298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.921985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:21.924580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-f7b42d17-daf7-48ec-926c-1b9ad97bc58b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:21.945184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:21.946271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:21.947153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:21.947153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:21.947153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:21.947957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 438/912] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:21.949231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7a26e-b8e5-44a1-897e-05ca1843a091 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:21.949517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7a26e-b8e5-44a1-897e-05ca1843a091 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:21.949791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7a26e-b8e5-44a1-897e-05ca1843a091 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:21.951178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 439/913] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:21.953523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7a26e-b8e5-44a1-897e-05ca1843a091 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:21.971863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:21.975770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48bd46-67a4-4009-8835-8e6817052811 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.011670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.012303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.012686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.012954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.013680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.013978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.016136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df5e9-7734-43c4-82dd-fa408b30be2c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.025988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 475/914] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:22.025988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.025988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.025988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.042156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:22.043449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:22.063711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7a26e-b8e5-44a1-897e-05ca1843a091 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.101335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.102182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.102603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.102875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.103555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.103833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.105214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.120177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 476/915] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:22.120177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.125085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 440/916] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:22.131050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.168506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:22.238530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:22.246615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298528-f652-4833-9ff9-4ca08a54c99b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.248763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ca7ce-5808-405a-9932-8c41718fc80c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.291071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.291805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.292200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.292552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.292877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.293720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.294420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.295034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.296868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.298519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.304470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.306151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:22.319341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:22.320327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:22.321413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 441/917] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:22.326434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.385735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 477/918] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:22.419869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.448188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:22.456198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791d5c-fa11-4048-af4e-dc3af7726f77 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.502532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.503625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.503625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.504005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.504913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.505317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.506878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.521906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 442/919] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:22.532879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.567234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9190-530d-450b-9dcb-621379488dcd None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.620334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.620978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.621258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.621535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.622177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.623550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:22.640206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:22.641289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:22.642278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 478/920] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:22.649074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.652418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05982f-05c8-4715-9f69-4a5d5b346f5c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.684573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:22.704463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.705405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.705856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.709034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.709034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] RBAC: Adding query filter params (role.id=307a932996254349baf41081822237c9, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=66f04604a2f44191a8af25bab59e44bc) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.709034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.710247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.719564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 443/921] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/role_assignments?user.id=66f04604a2f44191a8af25bab59e44bc&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=307a932996254349baf41081822237c9 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:22.764346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8ebb-71ae-4d0f-aeb6-595d63cc147d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.807005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.807639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.807898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.808213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.808869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.809170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.810593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.817095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 479/922] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:22.822734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:22.849949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:22.942701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a48ded-dd6c-4216-8db7-f7a1542cee78 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:22.978514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:22.979127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:22.979408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:22.979682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:22.980345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:22.980655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:22.982033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:22.988151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 444/923] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:22.993875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:23.077073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:23.099973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ce1d-6470-4c28-b035-fbc82acafac1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:23.136871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:23.137476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.137749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.138021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.138715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] RBAC: Adding query filter params (interface=public, service_id=fffd727628d949609c28e4dfd0aee464, region_id=RegionOne) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:23.139017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:23.140620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee8fe6-596e-47bb-b738-eed3d4bb74c9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:23.146271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 480/924] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:22 2022] GET /identity/v3/endpoints?service_id=fffd727628d949609c28e4dfd0aee464&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:23.151849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:23.187252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515557-b0a4-49a9-baa8-6c8f9957d37c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.187541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515557-b0a4-49a9-baa8-6c8f9957d37c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.187723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515557-b0a4-49a9-baa8-6c8f9957d37c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.189514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 481/925] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:23.198659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.198939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.199287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.219153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:23.220464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:23.247543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:23.273303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c64c1-0bf6-4cd3-b80c-1b72e17dc489 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:23.295434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 482/926] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:23.318575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:23.319552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.320104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.320616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.321512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:23.323219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:23.330530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 445/927] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:23 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:48:23.568932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:23.646420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:23.679067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104bf33-1c6b-4062-ae8b-bbe9a987d50c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:23.714763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:23.715403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.715685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.715962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.716584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:23.718588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:23.727335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:23.728298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:23.729209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 483/928] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Aug 25 02:48:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:23.735138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:23.866279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41a846-2f18-42a1-85d7-a3135dfb72ee None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:23.903525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:23.904616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:23.905139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:23.905643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:23.906516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:23.907036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:23.908713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:23.914785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 446/929] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:23.920481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:24.034194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c84e-4ef1-4573-b20d-863de7728136 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:24.069832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:24.070447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.070734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.071010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.071665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:24.071971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:24.073416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68559ec8-2d7d-41dd-bc2f-4a2e3db4090c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:24.074687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:24.079348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 484/930] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:24.084846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:24.098485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:24.197372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458b09-814d-4c78-badc-6a0b2716d951 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:24.232060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:24.232698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.232981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.233846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.234556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:24.235941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ef013-b750-4d7e-8fcb-c97305ce9522 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:24.349829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 447/931] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:24.481684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9a6b-6b4a-4b0e-be4e-cdc73133155e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.482374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9a6b-6b4a-4b0e-be4e-cdc73133155e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.482870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9a6b-6b4a-4b0e-be4e-cdc73133155e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.485373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 485/932] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:24.490917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566451-f575-4c02-8e84-b897529cb80e None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.491211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566451-f575-4c02-8e84-b897529cb80e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.491494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566451-f575-4c02-8e84-b897529cb80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.510202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c3566451-f575-4c02-8e84-b897529cb80e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:24.511407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c3566451-f575-4c02-8e84-b897529cb80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:24.583539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:24.598760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 448/933] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:24.805560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7ab1-f915-48f5-9440-077a95d85cc8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.805748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7ab1-f915-48f5-9440-077a95d85cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.806012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb7ab1-f915-48f5-9440-077a95d85cc8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.807592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 486/934] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:24.813269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:24.813560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:24.813837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:24.832737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:24.833933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:24.896439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb7ab1-f915-48f5-9440-077a95d85cc8 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:24.906489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 449/935] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:25.020624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb7ab1-f915-48f5-9440-077a95d85cc8 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.060217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.060756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.061041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.061317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.062865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.064321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.066976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING keystone.server.flask.common [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 25 02:48:25.107082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.sql.core [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 02:48:25.107082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 25 02:48:25.107082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [parameters: {'id': '27e39aed94164f8a82f19e6d8dbd96b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 25 02:48:25.107082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81785) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27e39aed94164f8a82f19e6d8dbd96b3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27e39aed94164f8a82f19e6d8dbd96b3. Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:48:25.115226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 25 02:48:25.116034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application yield resource Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise self.value Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application session.commit() Aug 25 02:48:25.117418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.flush() Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application compat.raise_( Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 02:48:25.119276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 02:48:25.121773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:48:25.123708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [parameters: {'id': '27e39aed94164f8a82f19e6d8dbd96b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.127966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:25.129968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27e39aed94164f8a82f19e6d8dbd96b3. Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.132106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 487/936] 10.222.0.75 () {60 vars in 1140 bytes} [Thu Aug 25 02:48:24 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:25.135225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.184245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.248687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.267904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac69f6-5d67-4e37-8232-62a6900bbccd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.300092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a257dec-cbee-4f38-8241-cbbcccf6f1d6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.314669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.315288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.315760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.316041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.325248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.327613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.339984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.340976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.341482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.341964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-3eb2cdab-aa2e-4364-9c03-70fac29be2b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:25.343764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:25.344739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:25.345725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:25.348352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 450/937] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:25.348491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.350148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f73c-2097-4f1a-83c0-7d36cdd4c7d2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.350566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f73c-2097-4f1a-83c0-7d36cdd4c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.350848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0f73c-2097-4f1a-83c0-7d36cdd4c7d2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.352139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 451/938] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:25.352926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0f73c-2097-4f1a-83c0-7d36cdd4c7d2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.362753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0c92657c-e605-416a-a332-9093aa80ed16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:25.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:25.370123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 488/939] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:25.371691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.372109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.372241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.392592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:25.394346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:25.461952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0f73c-2097-4f1a-83c0-7d36cdd4c7d2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.472199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 489/940] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:25.472199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.500032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.500699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.500981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.501260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.502004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:25.502313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.504066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.523237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 452/941] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:25.532478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.597715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd46059-a67a-41b0-904a-6a801c67356a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.635978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.636931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.637375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.637803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.638644 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:25.639108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.640976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.653636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 490/942] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:25.660187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.708549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.750730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.778058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b2208-6fcf-4393-a959-0bb1dfadc661 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.813287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.814176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.814626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.815047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.826131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.828409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:25.843614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:25.844652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:25.845563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:25.845563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:25.845563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:25.847842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 453/943] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:25.853016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.877624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91396b5-e056-455e-b627-ee6c45157f48 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:25.886800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.924332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:25.925451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:25.929361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:25.929361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:25.929361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:25.929361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:25.943624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:25.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:25.945638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:25.948230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 491/944] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:25.957744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:25.979423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:25.992316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe41d-ff3b-46e4-b650-23824e64c92b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.039032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.039805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.040135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.040514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.041255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:26.041637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.043116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.060838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 454/945] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:26.066740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.072346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8657-a56b-4873-9898-6eb0bce0583c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.115125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.115742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.116027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.116333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.117037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:26.117349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.119178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.126432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 492/946] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:26.131653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.180203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.198483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384697-58c7-4e99-83e2-b555ffc4102b None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.238821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.239759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.240043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.240358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.250629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.252594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.266044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2498fdf-2afd-4097-be87-ff1a373a4856 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:26.267490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:26.268529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:26.269509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:26.273627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 455/947] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:26.275306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.294781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.316014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.316591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.316922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.317193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.317822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:26.318138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.319551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.329486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 493/948] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:26.332171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.400299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc3a6a-62ce-49e4-94a1-5df5a5475dc7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.437974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.438958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.439447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.439882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.440820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:26.441307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.442924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7253a9-aa3b-4abd-af1f-16ac79199e6c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.454788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.458586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d58b4-c92b-4c21-bdd6-ac1cd4389108 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.461191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 456/949] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:26.472396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93072-6b49-4efa-9062-b98cd5339125 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.472396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93072-6b49-4efa-9062-b98cd5339125 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.472396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93072-6b49-4efa-9062-b98cd5339125 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.472396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 457/950] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:26.472396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93072-6b49-4efa-9062-b98cd5339125 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.503574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.504643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.505924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.506356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.507189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.508740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c545-8e7b-4609-b0d9-dc223ac12b28 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.539520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.570744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93072-6b49-4efa-9062-b98cd5339125 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.600405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 494/951] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:26.602074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.602651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.603157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.625114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.625861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.626047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.626332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.628518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:26.629740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:26.644552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.646601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:26.669899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:26.671024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:26.672345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:26.672345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:26.672345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:26.679271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 458/952] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:26.681008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.721824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 495/953] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:26.799556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1292c80-09cb-4e13-bd86-0ba9aa866516 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:26.816871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.835153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:26.835860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:26.836282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:26.836630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:26.837348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:26.837725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:26.839165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:26.852821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 459/954] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:26.858495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:26.964565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:26.973325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39909dd-ae77-48b9-9df1-ddac5baa267b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.008645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.009480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.009944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.010579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.018508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.020168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.022442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:27.032260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:27.033602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:27.034773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:27.036591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 496/955] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:27.041010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:27.140003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae209fde-6b1a-4788-8ca4-f4bc941f2e53 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.168757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201334ad-71ff-4eee-a404-bba4fba7a9a8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.178295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.178934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.179213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.179485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.180323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.181550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:27.192209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:27.193321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:27.194276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 460/956] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:27.199122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:27.221676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.222542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.222993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.223423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.224860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:27.225607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.227888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.245643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 497/957] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:27.249384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:27.252473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:27.257150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:27.325866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45494-82ad-434a-a6f7-202bd0aa8817 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.366953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c22ff1-ead7-4d5b-b6d6-6a9edab2a4e0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.370072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.370867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.371340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.371773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.372585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:27.373056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.374646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.381226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 461/958] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:27.386605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:27.407805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.408912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.409362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.409804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/66f04604a2f44191a8af25bab59e44bc/roles/307a932996254349baf41081822237c9` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.412381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=66f04604a2f44191a8af25bab59e44bc, role_id=307a932996254349baf41081822237c9)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.414318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45660015-d6fe-48a7-bdca-70455f380eaf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.470260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 498/959] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:27 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/66f04604a2f44191a8af25bab59e44bc/roles/307a932996254349baf41081822237c9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:27.504689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9110198-a9a0-4e42-91ba-ccbbd5a99cd8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:27.514132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:27.552653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:27.553667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.554275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.554754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.555648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:27.556265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:27.557938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d2522-f695-4f0c-9d05-459e3d95e81c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:27.564080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 462/960] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:27.583729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb731-757c-4eb6-8171-91d969108910 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.584083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb731-757c-4eb6-8171-91d969108910 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.584336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb731-757c-4eb6-8171-91d969108910 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.586123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 499/961] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:27.592059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:27.592411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:27.593499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:27.613561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:27.614802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:27.703651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 463/962] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:27.977393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb731-757c-4eb6-8171-91d969108910 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:28.046086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:28.086977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb731-757c-4eb6-8171-91d969108910 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:28.125724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:28.126336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:28.126616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:28.126889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:28.127540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:28.129039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:28.164251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.sql.core [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 02:48:28.164251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 25 02:48:28.164251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [parameters: {'user_id': '9755cd3c361442ccafecac365c973e6c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 25 02:48:28.164251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81785) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:48:28.171979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:48:28.173527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application yield resource Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application next(self.gen) Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.rollback() Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.force_reraise() Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise self.value Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application session.commit() Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 25 02:48:28.174510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._prepare_impl() Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.session.flush() Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._flush(objects) Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application compat.raise_( Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application flush_context.execute() Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rec.execute(self) Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 25 02:48:28.175425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 25 02:48:28.177272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result = self._query(query) Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application conn.query(q) Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application result.read() Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 25 02:48:28.179319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [parameters: {'user_id': '9755cd3c361442ccafecac365c973e6c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:28.185195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 25 02:48:28.188474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:28.190892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 500/963] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Aug 25 02:48:27 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:28.191826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:28.280875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:28.283979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce4fa1-c829-492f-bf0c-ef7c532ff4ed None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:28.318892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:28.319517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:28.319793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:28.320059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:28.327469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:28.328939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-9585af71-311a-4913-ac2f-7f3c70c14c67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:28.340394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:28.342645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 25 02:48:28.345058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:28.352176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 464/964] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:28.352360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:28.466131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098000-e293-49db-8a39-299b9d607cf3 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:28.493853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:28.503036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12572-4815-468a-9155-804759af7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:28.503643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12572-4815-468a-9155-804759af7f50 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:28.503930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12572-4815-468a-9155-804759af7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:28.504231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12572-4815-468a-9155-804759af7f50 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:28.513670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe12572-4815-468a-9155-804759af7f50 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:28.513670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe12572-4815-468a-9155-804759af7f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:28.513670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe12572-4815-468a-9155-804759af7f50 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:28.547276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 501/965] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Aug 25 02:48:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:28.718426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8a3a3-3af2-45ea-8075-0a754e152e48 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:28.718426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8a3a3-3af2-45ea-8075-0a754e152e48 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:28.718426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8a3a3-3af2-45ea-8075-0a754e152e48 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:28.718426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 465/966] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:28.727467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:28.727467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:28.727467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:28.745391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:28.747234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:28.822862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 502/967] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:29.089098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8a3a3-3af2-45ea-8075-0a754e152e48 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.197146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8a3a3-3af2-45ea-8075-0a754e152e48 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:29.231383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:29.232016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.232326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.232601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.233746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:29.235115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:29.243771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:29.244789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:29.245618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 466/968] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:29.252471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.352188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1b1a6-5ce5-4238-a749-0d9579df67b0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:29.366655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:29.386433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:29.390599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.390599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.390599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.390599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:29.390599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:29.391853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3734-7a57-4a89-8899-e1136e78073a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:29.401407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 503/969] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:29.402537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.429610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:29.504786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b33706-f356-4f7e-8283-6b3c11f07e25 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:29.515215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdda55-102b-4cfa-9cde-1106bea382d5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.515506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdda55-102b-4cfa-9cde-1106bea382d5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.515769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdda55-102b-4cfa-9cde-1106bea382d5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.517166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 504/970] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:29.522064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8998d-1720-4435-8529-be8714b9c7f7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.522356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8998d-1720-4435-8529-be8714b9c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.522633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8998d-1720-4435-8529-be8714b9c7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.540596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:29.541068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-83e8998d-1720-4435-8529-be8714b9c7f7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:29.541211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.541487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.541758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.542247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-83e8998d-1720-4435-8529-be8714b9c7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:29.542387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:29.542690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:29.544407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e2458-85ef-4ea1-a100-17b301b37d87 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:29.549325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 467/971] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:29.550534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6629-bbe1-455d-bf94-e9f57ffd618a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.550818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6629-bbe1-455d-bf94-e9f57ffd618a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.551092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd6629-bbe1-455d-bf94-e9f57ffd618a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.552431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 468/972] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:29.556898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd6629-bbe1-455d-bf94-e9f57ffd618a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.611959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 505/973] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:29.613229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.613505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.613774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.636101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:29.637359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:29.650676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:29.666000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd6629-bbe1-455d-bf94-e9f57ffd618a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:29.705138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:29.705744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.706020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.706296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.707003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] RBAC: Adding query filter params (interface=admin, service_id=fffd727628d949609c28e4dfd0aee464, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:29.707318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:29.708743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:29.712384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 506/974] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:29.719445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 469/975] 10.222.0.75 () {56 vars in 1249 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/endpoints?service_id=fffd727628d949609c28e4dfd0aee464&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:29.720537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.763645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:29.824653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53587668-06bb-4b1f-a2cf-6d3a425c366c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:29.859820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:29.860452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:29.860732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:29.861000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:29.861596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:29.862948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:29.869021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 507/976] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:48:29.887021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:29.987419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.009458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca74c4-4967-4058-9e1f-66efb6d4b909 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.050528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.051186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.051466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.051734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.063073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:30.067744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:30.068739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:30.069697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 470/977] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:30.074820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.096140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:30.100112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b557c-cfc6-4094-83ff-05ce334b3286 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.101958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:30.136974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.137508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.137785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.138062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.138709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.140725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.196225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 508/978] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:29 2022] POST /identity/v3/services => generated 242 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:30.196432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8007d67-7fcb-446e-81a1-46cfe70df282 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.247402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.248029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.248419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.249160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.249886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:30.250270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.251725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.266600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 471/979] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:30.267534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.378907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0867-1d4e-4f60-8504-fab3e0a44446 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.412877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.413427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.413702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.413968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.426847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.428341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:30.443537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:30.444891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:30.445817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:30.445817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:30.445817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:30.445817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 509/980] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:30.450674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.521447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:30.552847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aefd8c-993e-460f-ba6a-ae37bd42ae42 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.587757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.588395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.588673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.588943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.589574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:30.589875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.591231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.603332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 472/981] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:30.609471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.626380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:30.711934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b21797-ae8e-4c6b-b472-4180cb58f2a8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.749027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.750057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.750536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.751016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.758479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.760111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:30.772037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:30.773304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 25 02:48:30.774470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:30.776285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 510/982] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:30.784196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:30.885579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c85df7-33e3-4b81-945a-ec7f3280a64f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:30.921208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:30.927231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:30.927970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:30.928296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:30.931607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:30.931607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:30.931607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:30.931607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:30.953711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 473/983] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:30.955977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.036383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:31.061432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9b73-5c6e-4741-bb4d-dab9c78bcbd6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:31.095211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:31.095819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.096106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.096408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.163121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:31.164638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:31.239150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:31.270120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 511/984] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:30 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:31.271318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.374808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c489-4869-4efb-abec-5bf4e3e5f866 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:31.408593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:31.409223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.409507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.409781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.419993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:31.422243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:31.437191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:31.438170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:31.443807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:31.443807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:31.443807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:31.443807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 474/985] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:31.444119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.551756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc739-67a2-4ece-9b31-e2e770e71056 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:31.586743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:31.587352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.587632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.587904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.588584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:31.588889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:31.590271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:31.603306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 512/986] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:31.609673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.652274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:31.710815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2566b-1aec-4535-bc22-55b9b2521ab8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:31.745209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:31.745843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.746122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.746395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.753340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:31.754783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-ec73febe-41f4-45ea-b7d5-1375ef6ead72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:31.767219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:31.768431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:31.769395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:31.772463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 475/987] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:31.774037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.809555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:31.828967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944ee4-4110-458f-b545-e21af70db6a9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.829250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944ee4-4110-458f-b545-e21af70db6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.831609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944ee4-4110-458f-b545-e21af70db6a9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.831609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 476/988] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:31.836309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.836600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.836875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.855750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:31.857416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:31.883934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056cb09-9322-4254-b029-d2a80eaa4ae5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:31.923888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:31.924700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:31.924700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:31.924985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:31.925671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:31.925973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:31.927402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941ffa1-30a7-47d5-aeef-99c852c7f607 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:31.939205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 477/989] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:31.946058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 513/990] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:31.951940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:31.982045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.052676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e984e60-7707-432a-ad0a-40bf9fd9e29b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.086277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.086883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.087162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.087440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.088177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] RBAC: Adding query filter params (role.id=307a932996254349baf41081822237c9, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=66f04604a2f44191a8af25bab59e44bc) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:32.088497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.089923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b614db-7c84-4aba-b139-a55fb0ead8dc None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.100384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 478/991] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:31 2022] GET /identity/v3/role_assignments?user.id=66f04604a2f44191a8af25bab59e44bc&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=307a932996254349baf41081822237c9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:32.153722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411252-db78-40a9-8262-cd97661af630 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.154012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411252-db78-40a9-8262-cd97661af630 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.154290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411252-db78-40a9-8262-cd97661af630 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.156509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 514/992] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:32.162168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.162450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.162730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.182483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:32.183626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:32.210554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411252-db78-40a9-8262-cd97661af630 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.262132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 479/993] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:32.322548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.330686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411252-db78-40a9-8262-cd97661af630 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.367426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.368093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.368400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.368672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.369249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.370657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:32.379886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:32.380930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.385016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 515/994] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:32.387418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.414073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.492369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2e9ff-d297-4207-9b34-baec5d44d83e None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.526329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.526935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.527216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.527491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.528119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:32.528452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.530446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.534113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.540725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 480/995] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:32.541474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.635280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.642942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e055d-0aae-45af-aa6f-3c96a72fdb42 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.647127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a63-382b-4dc7-bf85-6b9e2aa29f83 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.681040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.681816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.682096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.682354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.682692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.682931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.683299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.683576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.683848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.684319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.684509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:32.684814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.686226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.691746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 481/996] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:32.694277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:32.695229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:32.696611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 516/997] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:32.701048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.701493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.728854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.805725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5587d9-fe9d-427e-8869-c7d4ee82ecf8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.806906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad94a5-89bf-4cec-8c57-060646cc511e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:32.839641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.840274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.840593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.840826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.841173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:32.841497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:32.841780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.841954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:32.842210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:32.843025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:32.843676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:32.844090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.845047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:32.849374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 517/998] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:32.854655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:32.940408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 482/999] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:32.944344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:32.961460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865b1c8-78a0-4e9b-8b22-909589c65a78 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:33.007365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:33.007989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:33.008422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:33.008708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:33.009358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:33.009668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:33.011074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:33.021595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 518/1000] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:33.022895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:33.040389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:33.135139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249b07-24d6-4707-9ff6-5e7f601e240f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:33.173342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:33.173963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:33.174244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:33.174516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:33.175220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] RBAC: Adding query filter params (interface=public, service_id=43f0ad6d24bc4fa4920b37c050136c7b, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:33.175530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:33.176924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83975742-1eab-405e-90af-0d26cd888582 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:33.193350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 483/1001] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:33 2022] GET /identity/v3/endpoints?service_id=43f0ad6d24bc4fa4920b37c050136c7b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:33.194944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:33.300161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf674-4c15-4a9c-91d2-93cd6b0bdd17 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:33.334090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:33.334710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:33.334946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:33.335220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:33.335817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:33.337196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4efc-5c25-4316-9ea6-d026837b67c4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:33.351917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 519/1002] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:33 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:48:34.083127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4c62-1758-4f57-b0fa-02db4dc9a8d1 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.083634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4c62-1758-4f57-b0fa-02db4dc9a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.083745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f4c62-1758-4f57-b0fa-02db4dc9a8d1 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.085078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 484/1003] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:34.090291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.090643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.090986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.109189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:34.110343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:34.113883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:34.184251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 520/1004] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:34.453696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f4c62-1758-4f57-b0fa-02db4dc9a8d1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:34.542147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:34.566509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f4c62-1758-4f57-b0fa-02db4dc9a8d1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:34.601242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:34.601872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.602153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.602454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] PATH_INFO: `/v3/services/image` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.603028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:34.604430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:34.613863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:34.615044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:34.615933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:34.619897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 485/1005] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Aug 25 02:48:34 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:34.621434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:34.722272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3cd1-0e33-4ea5-b550-87f8e334a6a1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:34.756717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:34.757331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.757610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.757885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.758515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] RBAC: Adding query filter params (name=image) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:34.758820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:34.760201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de703b59-a808-407f-955c-a78964f7df29 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:34.765627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 521/1006] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:34.771238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:34.877471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e9876-57b5-4b85-a775-646607f0fb18 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:34.888684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e657-679c-400d-ba70-3f9a1d5a0622 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.889538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e657-679c-400d-ba70-3f9a1d5a0622 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.889894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07e657-679c-400d-ba70-3f9a1d5a0622 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.891695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 522/1007] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:34.896655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb885c-916d-4abc-83fd-83e3f263a8a2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.896944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb885c-916d-4abc-83fd-83e3f263a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.897221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb885c-916d-4abc-83fd-83e3f263a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.905442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:34.917850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7ceb885c-916d-4abc-83fd-83e3f263a8a2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:34.919432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7ceb885c-916d-4abc-83fd-83e3f263a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:34.920996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:34.921798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:34.922086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:34.922363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:34.923005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:34.923322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:34.924779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:34.931864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 486/1008] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:34.998126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 523/1009] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:35.266790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:35.270645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:35.339853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b5a-77c2-4dc8-ba8a-aad0077a3bd8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:35.340208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b5a-77c2-4dc8-ba8a-aad0077a3bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:35.340432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b5a-77c2-4dc8-ba8a-aad0077a3bd8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:35.342074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 524/1010] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:35.347594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:35.347879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:35.348233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:35.369392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c3-5904-4726-840b-ab557529cdd7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:35.375075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:35.376328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:35.405618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:35.406246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:35.406524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:35.406820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:35.407881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:35.409348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:35.410900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 487/1011] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:35.416502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:35.454297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 525/1012] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:35.490417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:35.524032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70a005-255e-4859-8d73-5beb4cce6f9d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:35.558284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:35.558885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:35.559159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:35.559426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:35.560056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:35.562085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:35.660176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 488/1013] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:35 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:35.740797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:35.809142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:35.847390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb56a3a-7d92-47d7-b33c-56729581a8d2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:35.883123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:35.883753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:35.884028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:35.884356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:35.884944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:35.886278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:35.895305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:35.896355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:35.897826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 526/1014] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:35.902393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:36.009273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27a6f-7dc1-49b2-9cad-6e42fceeeaa4 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:36.034188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:36.046904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:36.047578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:36.047862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:36.048133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:36.055373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:36.055373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:36.055373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:36.055780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 489/1015] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:36.062008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:36.166106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9249-04be-41d1-a3c1-52546de58ee0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:36.192361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:36.200274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:36.200874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:36.201151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:36.201565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:36.202712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:36.203096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:36.204411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293517f-915b-42d0-8293-257a807b40fd None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:36.209327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 527/1016] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Aug 25 02:48:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:36.217352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:36.318638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d86c6-89dd-4c7f-82ca-583743f8c8fd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:36.352581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:36.353194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:36.353478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:36.353754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:36.354426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:36.355802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be984b96-0cb1-4c56-89a3-1e10292bbb9d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:36.415990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 490/1017] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:36.895504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ba2cc-0c7f-44d8-b0ca-099d30321764 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:36.896059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ba2cc-0c7f-44d8-b0ca-099d30321764 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:36.896059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ba2cc-0c7f-44d8-b0ca-099d30321764 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:36.897461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 528/1018] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:36.902783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604c43-fb91-42c2-8b52-841fb7f675fa None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:36.903082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604c43-fb91-42c2-8b52-841fb7f675fa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:36.903361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57604c43-fb91-42c2-8b52-841fb7f675fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:36.923309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-57604c43-fb91-42c2-8b52-841fb7f675fa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:36.924546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-57604c43-fb91-42c2-8b52-841fb7f675fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:36.968900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:37.001028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 491/1019] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:36 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:37.273144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ba2cc-0c7f-44d8-b0ca-099d30321764 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:37.377737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ba2cc-0c7f-44d8-b0ca-099d30321764 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:37.411760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:37.412401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:37.412685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:37.412960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:37.413614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:37.414970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341d434-915f-417a-98e6-0ccda6a25d85 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:37.490968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 529/1020] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:48:37 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:37.659351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8939e-c59c-477d-afe0-483e856716ee None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:37.659905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8939e-c59c-477d-afe0-483e856716ee None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:37.659905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8939e-c59c-477d-afe0-483e856716ee None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:37.661596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 492/1021] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:37.666821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:37.667109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:37.667385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:37.687177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:37.688525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:37.738208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:37.764920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 530/1022] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:38.038505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8939e-c59c-477d-afe0-483e856716ee None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:38.145908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8939e-c59c-477d-afe0-483e856716ee None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:38.181317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:38.181869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:38.182147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:38.182422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:38.183596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:38.185058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5886ea-1352-423a-bdae-9bc7adbe4a83 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:38.187176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 493/1023] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:38.192642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:38.290611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e598146-3b9f-48fd-adca-fd0ada7801ee None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:38.324872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:38.325481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:38.325760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:38.326034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:38.327300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:38.328755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:38.464629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba191d-6a07-468f-8129-51e0f48be12a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:38.464925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba191d-6a07-468f-8129-51e0f48be12a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:38.465111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba191d-6a07-468f-8129-51e0f48be12a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:38.466580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 494/1024] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:38.472207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 531/1025] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:48:38 2022] POST /identity/v3/users => generated 285 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:38.472388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:38.472479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:38.472680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:38.492567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:38.493856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:38.530945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:38.579486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 495/1026] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:38.854078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:38.890102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:38.965693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e7cb-0ef5-44a0-956a-b02111e29f9a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.002335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.003311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.003793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.004316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.014898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:39.020419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:39.021366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:39.022216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.024508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 532/1027] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:39.031365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.133880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c75d-a8e6-49c1-bdbc-b87717a50ba5 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.169199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.169840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.170108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.170514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.171353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:39.171661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.173074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.185663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 496/1028] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:39.187700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.284522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:39.292113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70f709-9eda-4304-81fa-980ce5b130cd None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.327698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.328452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.328737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.329015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.339309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.340836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-15231196-da42-4cb4-8eac-52d8c57d7ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:39.355357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:39.362168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:39.363090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:39.363090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:39.363090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.363090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 533/1029] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:39.363320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.395055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:39.429499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff01d-d13c-4543-913d-db6dd1d74978 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.429811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff01d-d13c-4543-913d-db6dd1d74978 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.430043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ff01d-d13c-4543-913d-db6dd1d74978 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.431513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 534/1030] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:39.436898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.437182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.437443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.455805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:39.457010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:39.470957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869583-6f9b-47ef-80c2-24efeb35abf2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.508763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.509491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.509770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.510040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.510703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:39.511007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.512545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.524664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 535/1031] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:39.524845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 497/1032] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:39.533569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.615566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:39.640122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91e5cd-5a0c-41ba-b436-b3b7be42415a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.678868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.679422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.679703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.679978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.687223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.688683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:39.700076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:39.706138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.707171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 536/1033] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:39.707795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.794410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.802994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:39.813142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3600d3-d5b0-4ab8-8b10-ad6847a5e2b9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.849017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.849623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.849906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.850179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.850903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:39.851211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.852687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.875364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 498/1034] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:39.876450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.900837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11660-76c4-49da-89dc-f55a587de0f1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:39.945045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:39.945917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:39.946375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:39.946809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] PATH_INFO: `/v3/services/image` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:39.947561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:39.949098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:39.952894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:39.960019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:39.961485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:39.962511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 537/1035] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:39.967969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:39.990935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad86fb-8280-4ffd-abac-430ebe418236 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.026083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.026697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.027030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.027303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.049738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.078676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521946b-e421-4650-9b49-6cf4353d48df None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.110446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.112264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.114816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.115454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.115713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.115981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.116648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] RBAC: Adding query filter params (name=image) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:40.116948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.118290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.124236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 538/1036] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:40.130114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.199643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.233218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 499/1037] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:40.239118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.244114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e28ad-3cfd-47b8-9ab8-29357590fffb None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.282312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.282927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.283204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.283473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.284196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:40.284503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.285863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.296914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 539/1038] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:40.297092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.298425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.347865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d4ab-44fc-4d68-bd09-9b252bffc41c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.382954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.383978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.384347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.384624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.397244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.398690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:40.413424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:40.419527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:40.420621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:40.420621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:40.420621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:40.420621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 500/1039] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:40.420924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.428514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7707b-ee3d-4e2e-9533-81d1877fd3e6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.460995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.471381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.472376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.472842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.473297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.474493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] RBAC: Adding query filter params (interface=public, service_id=0c9bc65d948243048492b496d162577e, region_id=RegionOne) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:40.475015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.477364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.487603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 540/1040] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/endpoints?service_id=0c9bc65d948243048492b496d162577e&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:40.489009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.531158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.534349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b51015-753e-40d3-9af6-3a6ab19fdc92 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.573523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.574128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.574402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.574671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.575312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:40.575616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.577725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc4f92-dd17-47fd-b9f6-3ca148d75fec None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.589289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 501/1041] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:40.590578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ab02-5008-4260-849f-9ca4026e8373 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.590860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ab02-5008-4260-849f-9ca4026e8373 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.591133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ab02-5008-4260-849f-9ca4026e8373 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.593225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 502/1042] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:40.597628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ab02-5008-4260-849f-9ca4026e8373 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.606901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2f6c-702b-407d-b957-40e50db0a110 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.647341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.648953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.650109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.650393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.651004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.652394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc480f-6c5b-4752-a1fc-d03af7f9c356 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.661435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 541/1043] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 02:48:40.662792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.663984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.664480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.691846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:40.693486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:40.714975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ab02-5008-4260-849f-9ca4026e8373 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.755534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.756209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.756504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.756793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.765371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.766008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.767512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:40.781550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:40.783305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:40.784271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 503/1044] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:40.790683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:40.799762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 542/1045] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:40.817509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:40.911210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352692-4d0f-4f27-8836-ad73639791ab None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:40.950189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:40.950805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:40.951072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:40.951863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:40.952589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:40.952894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:40.954417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:40.966407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 504/1046] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:40.972971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:41.076938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9233c2-2ead-4aab-af8a-2409a27458ec None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:41.077379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:41.114774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:41.115386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:41.115670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:41.115945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:41.116706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] RBAC: Adding query filter params (role.id=307a932996254349baf41081822237c9, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:41.117014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:41.118435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:41.124557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 543/1047] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:40 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=307a932996254349baf41081822237c9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:41.180372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:41.202363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360ca9-de55-4e51-9185-4cbf310dd9a9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:41.240016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:41.240614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:41.240888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:41.241159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:41.253370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:41.258179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:41.259099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:41.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 505/1048] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:41.266161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:41.320670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:41.372377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cbcea-a971-4f93-b3bb-6d4f0daaf393 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:41.407512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:41.408131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:41.408439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:41.408714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:41.409408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:41.409716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:41.411104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:41.425026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 544/1049] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:41.426210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:41.527389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5482-f340-49e1-82bc-c2961b2791cb None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:41.562079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:41.564239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:41.564239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:41.564239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] PATH_INFO: `/v3/users/d4442a944b9a4adc9904748fb6afbbfa` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:41.575357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:41.580198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] RBAC: Authorizing `identity:get_user(user_id=d4442a944b9a4adc9904748fb6afbbfa)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:41.580198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:41.582058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 506/1050] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:41 2022] GET /identity/v3/users/d4442a944b9a4adc9904748fb6afbbfa => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:41.587886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:41.693102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7a468-d134-418f-9d3d-ebea4b99828f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:41.707175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:41.728506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:41.729383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:41.729816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:41.730259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:41.803744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:41.805175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:41.907421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 545/1051] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:41.913088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:42.022111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87ec27-5abe-43f7-af67-24b149ee0197 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:42.056534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:42.057098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:42.057375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:42.057648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:42.058756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:42.060360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a82e19-1251-4217-8f47-75a27991aee7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:42.067058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 507/1052] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:48:41 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:42.068177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:42.082770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:42.168950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd630b2-b867-442f-9646-d349e13315f1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:42.204797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:42.205427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:42.205708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:42.205982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:42.206724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=37002f608fb449b68592e52a264bec1f, user.id=d4442a944b9a4adc9904748fb6afbbfa) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:42.207036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:42.208485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52d222-0d48-4dbe-adb3-46a6a3525263 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:42.220188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 546/1053] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:42 2022] GET /identity/v3/role_assignments?user.id=d4442a944b9a4adc9904748fb6afbbfa&scope.project.id=37002f608fb449b68592e52a264bec1f&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:42.652525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9f35c-e292-4dba-8386-c380f676eb7e None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:42.653088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9f35c-e292-4dba-8386-c380f676eb7e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:42.653088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9f35c-e292-4dba-8386-c380f676eb7e None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:42.655188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 508/1054] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:42.660472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85e4b1-c805-4ef5-94bd-ed6299c05a4d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:42.660924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85e4b1-c805-4ef5-94bd-ed6299c05a4d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:42.661003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85e4b1-c805-4ef5-94bd-ed6299c05a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:42.680353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-dd85e4b1-c805-4ef5-94bd-ed6299c05a4d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:42.681533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-dd85e4b1-c805-4ef5-94bd-ed6299c05a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:42.750226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 547/1055] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:43.022907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9f35c-e292-4dba-8386-c380f676eb7e None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.040048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c540fb4-42bf-4e08-83b8-9d05ea1a474c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.040377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c540fb4-42bf-4e08-83b8-9d05ea1a474c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.040655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c540fb4-42bf-4e08-83b8-9d05ea1a474c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.042007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 548/1056] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:43.047777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.048063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.048368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.061620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.068134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:43.069442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:43.104410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.141376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 549/1057] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:43.143507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9f35c-e292-4dba-8386-c380f676eb7e None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.181103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.181649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.181925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.182195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] PATH_INFO: `/v3/services/image` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.182780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.184251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:43.193560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:43.194571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:43.195416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 509/1058] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:43.200624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.303578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c82637-7399-490f-a90f-ff1925aa3b31 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.337323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.337922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.338197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.338463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.339085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] RBAC: Adding query filter params (name=image) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:43.339383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.340758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.342822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.351530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 550/1059] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:43.353687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.419244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.462831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537765a9-14ed-477c-8e0d-1f407885586e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.496213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.502579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.503299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.503580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.503851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.504515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:43.504821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.506696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.516549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 510/1060] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:43.517807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.530591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8de9bb-68e6-49c0-84e4-cf4f2a91f83e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.566945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.567570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.567849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.568118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.580097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:43.585749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:43.586752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:43.587602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 551/1061] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:43.593334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.632888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b3a-d43f-4253-a965-76de23dc3ba1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.661389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.669907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.671055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.671478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.671760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.672442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.673827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.710101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b399-0f4d-49f5-9736-b124f8401b81 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.745671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.746199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.746474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.746745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.747406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:43.747711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.749609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.762422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 552/1062] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:43.762607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 511/1063] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:48:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:48:43.764202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:43.876573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe032f-554d-4c48-8415-eb9906970f02 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:43.901215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:43.923145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-14031826-ce08-4adf-85a6-0facd90a019d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:43.966352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:43.968528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:43.969463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:43.970578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 553/1064] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:43.970660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.019498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.072664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ed2af-2347-4a4e-8449-0ae498085ea9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.098935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7160-4a73-45bf-b33a-664529da6501 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.099212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7160-4a73-45bf-b33a-664529da6501 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.099484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7160-4a73-45bf-b33a-664529da6501 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.100862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 554/1065] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:44.105564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.105845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.106119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.111623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.112696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.114024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.114491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.115358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:44.115861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.117439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.124349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:44.125521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:44.134126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 512/1066] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:44.141265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.203704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 555/1067] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:44.246906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a758d7-b881-406e-94ea-cc6727452377 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.261359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.285954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.286506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.286784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.287055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.294422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.296474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:44.309399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:44.310358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:44.317034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 513/1068] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:44.317628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.417734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.421986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b340fd-49b3-48bd-a389-2f6e2367c815 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.457342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.457953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.458230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.458498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.459219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:44.459525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.460968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.474463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.484380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 556/1069] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:44.485462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.576668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbd8f-5bdb-4de8-a09b-327d32af5e57 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.596368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c40a89-b61d-49ee-b283-626f24a1545e None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.613218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.613841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.614162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.614383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.625151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:44.630560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:44.635816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:44.636939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 514/1070] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:44.638078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.639244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.639524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.639791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.641237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.650811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.652265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:44.667556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:44.669395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:44.670900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:44.670900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:44.670900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:44.670900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 557/1071] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:44.674708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.680627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.690958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.754527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d423e-2958-4002-8ffb-3051fc92b1e2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.786812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ca803-fbe1-4758-a66d-310e933306ad None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.792754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.793615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.793891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.794160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.794817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:44.795341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.797178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.805475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 515/1072] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:44.811113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.836549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.838034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.838501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.839099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.840316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:44.841115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.844578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.861274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 558/1073] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:44.867462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:44.925884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.944420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3660e-b853-4612-9dea-54b582d7cac8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:44.975461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:44.989422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:44.990054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:44.990329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:44.990600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] PATH_INFO: `/v3/users/d4442a944b9a4adc9904748fb6afbbfa` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:44.991777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] RBAC: Authorizing `identity:get_user(user_id=d4442a944b9a4adc9904748fb6afbbfa)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:44.993243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:44.995395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 516/1074] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/users/d4442a944b9a4adc9904748fb6afbbfa => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:45.004213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:45.018984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0034f-6baa-48d9-af08-4453d76dc8b4 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:45.066044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:45.066933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.067367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.067788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.075684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:45.077974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:45.090169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:45.091262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:45.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:45.093029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 559/1075] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:45.098518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:45.137954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17c57c-8fc8-448d-bf10-15b6e2a4dbf2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:45.177231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:45.177863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.178141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.178412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.179530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:45.181010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:45.183196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 517/1076] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:48:45 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:45.191600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:45.224970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:45.245614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80933447-2bdd-4747-873e-7b36ed0f50ab None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:45.287612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:45.293144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:45.294042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.294355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.294553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.295784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:45.296052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:45.297608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:45.310063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 560/1077] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:45.316264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:45.318666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39970ae-afd2-4e66-a339-46911546b74e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:45.357052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:45.358193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.358473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.358825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f/users/d4442a944b9a4adc9904748fb6afbbfa/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.360823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=37002f608fb449b68592e52a264bec1f, user_id=d4442a944b9a4adc9904748fb6afbbfa, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:45.362331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e046-e535-4773-b07a-e3737e054f9b None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:45.419491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1bf2e-7710-4bc4-8f2f-ae014c1b4cf1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:45.438924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 518/1078] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:45 2022] PUT /identity/v3/projects/37002f608fb449b68592e52a264bec1f/users/d4442a944b9a4adc9904748fb6afbbfa/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:45.457847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:45.458528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.458807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.459080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/307a932996254349baf41081822237c9` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.461558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=867d21f5f80d4cc48cc53e0b89c7f2f8, role_id=307a932996254349baf41081822237c9)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:45.463019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cc204-fd8d-4715-8ba7-e6bb18c35f2c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:45.536715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 561/1079] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:45 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/867d21f5f80d4cc48cc53e0b89c7f2f8/roles/307a932996254349baf41081822237c9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 02:48:45.777642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ade0b-b0df-4568-b6d9-1708d46521fc None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.778160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ade0b-b0df-4568-b6d9-1708d46521fc None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.778240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ade0b-b0df-4568-b6d9-1708d46521fc None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.780081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 519/1080] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:45.785424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:45.785854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:45.786047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:45.805497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:45.806686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:45.852841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:45.874322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 562/1081] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:46.145015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ade0b-b0df-4568-b6d9-1708d46521fc None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:46.233824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:46.246692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ade0b-b0df-4568-b6d9-1708d46521fc None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:46.281523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:46.282125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:46.286545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:46.286545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:46.350281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:46.351730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:46.439984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:46.453454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 520/1082] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:46.459022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:46.565246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aa51a-8d56-4404-a626-a6ecf4d56cdf None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:46.601221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:46.601863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:46.602193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:46.602502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:46.613496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:46.614948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-3059a463-d500-410d-85f9-6d29de172f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:46.630430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:46.631474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:46.632824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:46.632824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:46.632824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:46.636236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 563/1083] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:46.638169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:46.738472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94887-8f1e-4ab3-96a9-13bc08ad0f72 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:46.772364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:46.772967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:46.773249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:46.773522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:46.774159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:46.774467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:46.775835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd2846-d101-4182-8c98-4c350c13e749 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:46.792254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 521/1084] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:47.411525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad63ded-35a2-480b-8362-0209cdfe80d1 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.412052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad63ded-35a2-480b-8362-0209cdfe80d1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.412052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad63ded-35a2-480b-8362-0209cdfe80d1 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.413436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 564/1085] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:47.418915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.419205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.419482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.438129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:47.439309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:47.463438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48deb723-8d5b-4614-b3fa-ea5cbeea01cf None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.463721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48deb723-8d5b-4614-b3fa-ea5cbeea01cf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.464018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48deb723-8d5b-4614-b3fa-ea5cbeea01cf None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.465378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 565/1086] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:48:47.471194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.471549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.471894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.483866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:47.490115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:47.491315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:47.507357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 522/1087] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:47.517575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:47.557425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 566/1088] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:48:47.781273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:47.826901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:47.881252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fa0e6-4ac4-48aa-9b8d-3236ca8ea84f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:47.916382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:47.917367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.917653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.918080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.926463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa452b-c801-4543-b355-6a3f84d400ed None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:47.933066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:47.938832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:47.939795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:47.940702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 523/1089] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:47.945792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:47.967993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:47.968651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:47.968930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:47.969211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:47.979057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:48:47.983978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:47.984957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:47.985892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 567/1090] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Aug 25 02:48:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:47.991166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:47.991938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.016342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.052672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b002d4b-8a15-4821-8493-b1fe530602f6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.086971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.087572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.087860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.088134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.088822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:48.089130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.090507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.090607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b785c-f5a6-4fbb-b3a3-eae05c9c5fe0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.098791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 524/1091] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.104463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.126525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.127140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.127419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.127694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.128499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:48.128806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.130196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.143421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 568/1092] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:48:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.144702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.200590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.203926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d1147-6b6b-485b-b418-d0fce45bd730 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.229405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.237595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.238563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.238868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.239496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] PATH_INFO: `/v3/users/d4442a944b9a4adc9904748fb6afbbfa` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.240659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=d4442a944b9a4adc9904748fb6afbbfa)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.242090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.243649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 525/1093] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/users/d4442a944b9a4adc9904748fb6afbbfa => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.244745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922a42-cd5d-43ae-9f91-19e1d053729b None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.250145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.279524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.280163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.280451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.280727 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.291036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.292496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:48.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:48.308001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:48.308923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:48.308923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:48.308923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:48.308923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 569/1094] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:48.314323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.350752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41a61-1c2f-4bed-b7b3-041f01805cb3 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.385774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.386371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.386653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.386925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.414414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44691ac-1881-4436-8124-9ea084bb3272 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.422137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.448501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.449101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.449381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.449655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.450294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:48.450596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.452814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.456110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.456624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.457681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.467086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 570/1095] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.477599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.557850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 526/1096] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 02:48:48.564607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.579324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d280ed-ca72-4797-827b-517563b900e9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.614179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.614809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.615088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.615359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.622405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.624975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.631084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:48:48.637303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:48.638350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:48:48.639669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 571/1097] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:48.644111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.663220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e1cc4-53ad-4074-8e86-484f86e0ba24 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.696781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.697410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.697690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.697965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.699093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.700570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.702688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 527/1098] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:48.708370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.717782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.768197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55b22f-55d8-47ce-961d-d1c854d028f7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.802315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.802918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.803197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.803477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.804235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:48.804545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.805960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.809097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5b7e9-a6e7-4d84-8d8d-62c9fd56fdfa None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.822852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 572/1099] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.828764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:48.842266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.842878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.843206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.843971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:48.844667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=37002f608fb449b68592e52a264bec1f, user.id=d4442a944b9a4adc9904748fb6afbbfa) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:48.844976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:48.846415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:48.849221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.856827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 528/1100] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/role_assignments?user.id=d4442a944b9a4adc9904748fb6afbbfa&scope.project.id=37002f608fb449b68592e52a264bec1f&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:48.949200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a4575-9f77-40f6-b6db-05ce3e382709 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:48.964064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:48.985558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:48.986174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:48.986450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:48.986717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.058023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.059596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.158734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 573/1101] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 25 02:48:49.164356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:49.261175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b4ac5-07c9-4393-8899-68b17abbae67 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:49.294147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:49.294957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:49.295402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:49.295496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.305638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.307067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:48:49.320866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:49.321981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:48:49.323071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:48:49.323071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:48:49.323071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:49.324295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 529/1102] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:48:49.328094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:49.343553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:49.429363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147ab2-e801-4e43-a8a7-d0285c0ef7b7 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:49.463311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:49.463907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:49.464216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:49.464493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.465131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:49.465438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.466834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.477892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 574/1103] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:49.483464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:49.535787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:49.583112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9f931-7dc0-494b-b176-018e41f4c8eb None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:49.615847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:49.616498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:49.616780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:49.617052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.624448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.625880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:48:49.637116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:48:49.638265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:48:49.639276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:48:49.640296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 530/1104] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:48:49.644220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:49.726431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:49.762165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1face-f82b-4fb9-b870-06ed629b6292 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:49.795226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:49.797989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:49.797989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:49.797989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.797989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:49.797989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.798870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e53e-e451-4481-9e8f-07abcd388152 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.810706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 575/1105] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:49.816931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:49.916209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb3ff8-e2a7-4012-a983-721910130696 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:49.930308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:49.952450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:49.953041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:49.953325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:49.953596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:49.954345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] RBAC: Adding query filter params (role.id=307a932996254349baf41081822237c9, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=867d21f5f80d4cc48cc53e0b89c7f2f8) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:49.954655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:49.956085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceab0e-e3bb-4774-968a-90b503d3b94c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:49.962466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 531/1106] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:49 2022] GET /identity/v3/role_assignments?user.id=867d21f5f80d4cc48cc53e0b89c7f2f8&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=307a932996254349baf41081822237c9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:48:50.815352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724683-b792-49cc-9123-64cb02794f1a None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:50.815868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724683-b792-49cc-9123-64cb02794f1a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:50.815986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724683-b792-49cc-9123-64cb02794f1a None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:50.817287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 576/1107] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:50.822330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:50.822710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:50.822879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:50.842387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:50.843686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:50.908984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 532/1108] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:51.174571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37724683-b792-49cc-9123-64cb02794f1a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:51.273794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37724683-b792-49cc-9123-64cb02794f1a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:51.306670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:51.307281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:51.307559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:51.307827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:51.308948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:51.310996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662f9a2-81d5-4e82-a955-d766e8d726b2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:51.312651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 577/1109] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:48:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:51.318212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:51.413281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b59ca-1fad-47d7-ad84-1234948503ee None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:51.421284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:51.447036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:51.447576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:51.447855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:51.448125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:51.448797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:51.450225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67424d0-4fcd-445d-928b-6ff3a58ef199 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:51.546517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 533/1110] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:48:51 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:48:53.480631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d60c-2b18-4afa-b31e-6f81613c4331 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:53.481193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d60c-2b18-4afa-b31e-6f81613c4331 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:53.481193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d60c-2b18-4afa-b31e-6f81613c4331 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:53.483252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 578/1111] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:53.488443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:53.488863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:53.488994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:53.506421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:53.507587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:53.571168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 534/1112] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:53.844460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558d60c-2b18-4afa-b31e-6f81613c4331 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:53.847283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:53.947381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558d60c-2b18-4afa-b31e-6f81613c4331 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:54.068077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:54.068796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:54.069070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:54.069336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] PATH_INFO: `/v3/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:54.070725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] RBAC: Authorizing `identity:get_role(role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:54.072073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:54.073421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 579/1113] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:53 2022] GET /identity/v3/roles/ee78bca73e4446e2b38a9e195677f064 => generated 223 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:54.079949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:54.171915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:54.180281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec98458-32d2-462e-a826-4aff878db459 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:54.213357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:54.213997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:54.214270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:54.214538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] PATH_INFO: `/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:54.227862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] RBAC: Authorizing `identity:get_user(user_id=64a9bcd1ed49444bbeef27b1bdf7594b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:54.229317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:54.230973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 535/1114] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:54 2022] GET /identity/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:54.237646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:54.305297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:54.335391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d667f7-7c78-41af-a95f-a01242fbcef8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:54.368866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:54.369486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:54.369761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:54.370029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:54.437260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:54.438629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:54.502817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:54.532878 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 580/1115] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:48:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:48:54.538795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:54.633897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e28ffd-83fe-4271-ba23-8173a12e6e52 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:54.666084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:54.666690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:54.666963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:54.667232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:54.668606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:54.670059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369552a-edda-4e8c-b63f-518d6e78837d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:54.672239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 536/1116] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:48:54 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:54.678280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:54.773121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4636-f956-4717-8719-b4886a353384 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:54.811979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:54.812595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:54.812872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:54.813146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:54.813862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=37002f608fb449b68592e52a264bec1f, user.id=64a9bcd1ed49444bbeef27b1bdf7594b) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:48:54.814171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:54.815613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a824c-c5c1-4363-96e5-7aed8f0f74da None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:54.820861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 581/1117] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:48:54 2022] GET /identity/v3/role_assignments?user.id=64a9bcd1ed49444bbeef27b1bdf7594b&scope.project.id=37002f608fb449b68592e52a264bec1f&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:56.722349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c844f-f7f0-4246-b198-0a0609cb6c11 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:56.722952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c844f-f7f0-4246-b198-0a0609cb6c11 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:56.722952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c844f-f7f0-4246-b198-0a0609cb6c11 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:56.724281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 537/1118] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:56.729479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:56.729899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:56.730031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:56.748193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:56.749991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:56.775423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:56.815819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 582/1119] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:57.089696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c844f-f7f0-4246-b198-0a0609cb6c11 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:57.108886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:57.190804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c844f-f7f0-4246-b198-0a0609cb6c11 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:57.224796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:57.225410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:57.225689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:57.225961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] PATH_INFO: `/v3/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:57.227334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] RBAC: Authorizing `identity:get_role(role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:57.228729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:57.230009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 538/1120] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:57 2022] GET /identity/v3/roles/ee78bca73e4446e2b38a9e195677f064 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:57.236496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:57.332853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07fa8e-9ee6-4511-9ad9-31525dcc7591 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:57.365454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:57.366070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:57.366343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:57.366611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] PATH_INFO: `/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:57.367699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=64a9bcd1ed49444bbeef27b1bdf7594b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:57.369141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:57.370687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 583/1121] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:57 2022] GET /identity/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:57.377213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:57.459393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:57.473435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8cd18-a744-46c5-bf66-ca5b0450c88d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:57.506699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:57.507308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:57.507583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:57.507854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:57.508972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:57.510382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e447e-a085-431b-ae48-dc53a8a18863 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:57.511969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 539/1122] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:48:57 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:48:57.518017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:48:57.548785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:48:57.617089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def08157-d16d-4e9b-894d-4c5f559db3a9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:48:57.649935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:48:57.650577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:57.650857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:57.651130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f/users/64a9bcd1ed49444bbeef27b1bdf7594b/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:57.653250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] RBAC: Authorizing `identity:create_grant(project_id=37002f608fb449b68592e52a264bec1f, user_id=64a9bcd1ed49444bbeef27b1bdf7594b, role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:48:57.654648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619ca9f-1239-49ac-bde9-808293e87e92 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:48:57.745332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 584/1123] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:48:57 2022] PUT /identity/v3/projects/37002f608fb449b68592e52a264bec1f/users/64a9bcd1ed49444bbeef27b1bdf7594b/roles/ee78bca73e4446e2b38a9e195677f064 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:48:59.617715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412abaf-8adf-45d1-b21a-59b66610a41a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:59.618270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412abaf-8adf-45d1-b21a-59b66610a41a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:59.618270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412abaf-8adf-45d1-b21a-59b66610a41a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:59.619599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 540/1124] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:48:59.625054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:48:59.625497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:48:59.625599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:48:59.644015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:48:59.645231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:48:59.709819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 585/1125] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:48:59.984273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412abaf-8adf-45d1-b21a-59b66610a41a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:00.085291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412abaf-8adf-45d1-b21a-59b66610a41a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:00.117764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:00.118379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:00.118657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:00.118930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] PATH_INFO: `/v3/roles/ee78bca73e4446e2b38a9e195677f064` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:00.120345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] RBAC: Authorizing `identity:get_role(role_id=ee78bca73e4446e2b38a9e195677f064)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:00.121703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:00.122997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 541/1126] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:48:59 2022] GET /identity/v3/roles/ee78bca73e4446e2b38a9e195677f064 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:00.129603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:00.153378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:00.226625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74201c99-5864-4e69-a69c-6a45fc3e750d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:00.259899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:00.260539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:00.260820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:00.261091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] PATH_INFO: `/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:00.262190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] RBAC: Authorizing `identity:get_user(user_id=64a9bcd1ed49444bbeef27b1bdf7594b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:00.263600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:00.265317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 586/1127] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:00 2022] GET /identity/v3/users/64a9bcd1ed49444bbeef27b1bdf7594b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:00.271119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:00.295194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:00.367873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128d784-5a74-43ff-90f5-ffb7143f7b32 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:00.401762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:00.402356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:00.402636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:00.402907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:00.470755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:00.472227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:00.493834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:00.566015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 542/1128] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:00.572206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:00.659873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:00.669189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b5ed9-e174-4bbc-8371-a51d9b175c07 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:00.703632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:00.704268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:00.704549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:00.704824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] PATH_INFO: `/v3/projects/37002f608fb449b68592e52a264bec1f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:00.705918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] RBAC: Authorizing `identity:get_project(project_id=37002f608fb449b68592e52a264bec1f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:00.707330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb671e7-9699-4a00-83ff-7c26afd5a15e None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:00.708961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 587/1129] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:00 2022] GET /identity/v3/projects/37002f608fb449b68592e52a264bec1f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:00.714977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:00.812093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9391-99d3-4892-9063-2636ce0456c5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:00.834002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:00.845153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] RBAC: Adding query filter params (role.id=ee78bca73e4446e2b38a9e195677f064, scope.project.id=37002f608fb449b68592e52a264bec1f, user.id=64a9bcd1ed49444bbeef27b1bdf7594b) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:00.856600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ee0b-86d2-49bd-8a50-858111054539 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:00.857144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 543/1130] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:49:00 2022] GET /identity/v3/role_assignments?user.id=64a9bcd1ed49444bbeef27b1bdf7594b&scope.project.id=37002f608fb449b68592e52a264bec1f&role.id=ee78bca73e4446e2b38a9e195677f064 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:02.802981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4704-8ef5-4bae-bce0-a12f91cedefc None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:02.803535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4704-8ef5-4bae-bce0-a12f91cedefc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:02.803535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4704-8ef5-4bae-bce0-a12f91cedefc None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:02.804899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 588/1131] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:02.810126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:02.810554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:02.810681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:02.828810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:02.829982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:02.894243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 544/1132] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:03.162340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf4704-8ef5-4bae-bce0-a12f91cedefc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:03.262600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf4704-8ef5-4bae-bce0-a12f91cedefc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:03.295806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:03.296442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:03.296719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:03.296986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:03.298086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:03.299491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5049f-37f9-48ea-adae-74d7461cb669 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:03.301059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 589/1133] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:03.306817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:03.401059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366078ba-4cda-4b51-9fbe-3e7a81c6ff6c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:03.417177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:03.434154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:03.434743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:03.435021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:03.435293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:03.435969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:03.437423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0f143-4b39-40fe-bae9-edd38f68560c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:03.552667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 545/1134] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:49:03 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 02:49:05.489987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bcbd2-6e85-4c07-bdee-33acd202da74 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:05.490519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bcbd2-6e85-4c07-bdee-33acd202da74 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:05.490519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bcbd2-6e85-4c07-bdee-33acd202da74 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:05.491877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 590/1135] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:05.497097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:05.497476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:05.497652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:05.515879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:05.517106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:05.581827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 546/1136] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:05.856719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bcbd2-6e85-4c07-bdee-33acd202da74 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:05.883696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:05.959948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bcbd2-6e85-4c07-bdee-33acd202da74 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:05.994525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:05.995149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:05.995432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:05.995706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:05.996788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:05.998236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf623d6-8c92-409c-8feb-80377bf1629b None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:05.999897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 591/1137] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:49:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:06.005612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:06.098247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:06.101674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0a48-57a3-40b3-9302-bdcd03d71162 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:06.134498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:06.135306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:06.135888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:06.136238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:06.136889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:06.138330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcd7e1-3e7a-4260-a803-462925e73b4f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:06.220559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 547/1138] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:49:06 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:49:08.178308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf5e98-07d4-4b07-8613-22584bf7aaa6 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:08.178860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf5e98-07d4-4b07-8613-22584bf7aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:08.178860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf5e98-07d4-4b07-8613-22584bf7aaa6 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:08.180250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 592/1139] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:08.185335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:08.185762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:08.185970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:08.203671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:08.204898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:08.270050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 548/1140] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:08.539174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf5e98-07d4-4b07-8613-22584bf7aaa6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:08.629973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:08.637567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf5e98-07d4-4b07-8613-22584bf7aaa6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:08.672118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:08.672760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:08.673039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:08.673313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:08.683075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:08.688446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:08.689597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:08.690676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 593/1141] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:08.695482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:08.778211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:08.793824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadef14-ddec-415a-b713-2e00d3d47036 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:08.828231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:08.828829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:08.829109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:08.829381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:08.830047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:08.830350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:08.831719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:08.839460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 549/1142] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:08.845976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:08.943933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713f65-9187-4e6b-b8f6-8634ca0a4436 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:08.976928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:08.977545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:08.977823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:08.978094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] PATH_INFO: `/v3/users/61f9855b1e674ce0b6234b30a6e8c187` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:08.991420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] RBAC: Authorizing `identity:get_user(user_id=61f9855b1e674ce0b6234b30a6e8c187)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:08.992875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:08.994451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 594/1143] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:08 2022] GET /identity/v3/users/61f9855b1e674ce0b6234b30a6e8c187 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:09.002109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:09.098646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115fd1f-86f8-431e-b764-34d368f0a0d8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:09.130637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:09.131227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:09.131503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:09.131775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:09.146873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:09.196689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:09.198125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:09.291313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 550/1144] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:09.298022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:09.313192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:09.397280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de745-bb21-4d59-987d-457c3ed64d83 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:09.430723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:09.431343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:09.431622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:09.431895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] PATH_INFO: `/v3/projects/f617faf352cd4576bb066475dd70fa20` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:09.433022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] RBAC: Authorizing `identity:get_project(project_id=f617faf352cd4576bb066475dd70fa20)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:09.434435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e7d91-e8b4-46ac-96b5-6a17ab62d4f6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:09.437108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 595/1145] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:09 2022] GET /identity/v3/projects/f617faf352cd4576bb066475dd70fa20 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:09.443218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:09.488914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:09.540342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cb49f-880a-4c96-abfb-7b7017c5b32e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:09.573449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:09.574041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:09.574323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:09.574597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:09.575314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=f617faf352cd4576bb066475dd70fa20, user.id=61f9855b1e674ce0b6234b30a6e8c187) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:09.575621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:09.577105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9422b-6fc8-4e0e-95cd-3606c18de94d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:09.582297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 551/1146] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:49:09 2022] GET /identity/v3/role_assignments?user.id=61f9855b1e674ce0b6234b30a6e8c187&scope.project.id=f617faf352cd4576bb066475dd70fa20&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:11.455088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93838-7bb3-4c74-a5e4-554a2fb34191 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:11.455613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93838-7bb3-4c74-a5e4-554a2fb34191 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:11.455613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93838-7bb3-4c74-a5e4-554a2fb34191 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:11.456992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 596/1147] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:11.461998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:11.462422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:11.462551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:11.480850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:11.482030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:11.547131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 552/1148] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:11.821071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f93838-7bb3-4c74-a5e4-554a2fb34191 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:11.899661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:11.919572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f93838-7bb3-4c74-a5e4-554a2fb34191 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:11.953412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:11.954021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:11.954295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:11.954563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:11.964199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:11.969479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:11.970590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:11.971519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 597/1149] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:11.976534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:12.018006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:12.074920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505088a6-8248-4668-8ba6-8ab22671a748 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:12.109414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:12.110014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:12.110294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:12.110567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:12.111226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:12.111530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:12.112923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:12.120639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 553/1150] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:12.127186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:12.224540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc7cc2-ba58-4b18-9af3-89e2b821365f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:12.257979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:12.258593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:12.258872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:12.259144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] PATH_INFO: `/v3/users/61f9855b1e674ce0b6234b30a6e8c187` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:12.260268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] RBAC: Authorizing `identity:get_user(user_id=61f9855b1e674ce0b6234b30a6e8c187)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:12.261685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:12.263237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 598/1151] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:12 2022] GET /identity/v3/users/61f9855b1e674ce0b6234b30a6e8c187 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:12.268893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:12.365261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb01dd-a0a3-49da-b635-74cbdf5664c5 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:12.366730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:12.399096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:12.399714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:12.399990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:12.400288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] PATH_INFO: `/v3/projects/f617faf352cd4576bb066475dd70fa20` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:12.401380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] RBAC: Authorizing `identity:get_project(project_id=f617faf352cd4576bb066475dd70fa20)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:12.402799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb4a1b-de88-4afb-8712-75a36f5b401a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:12.404530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 554/1152] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:12 2022] GET /identity/v3/projects/f617faf352cd4576bb066475dd70fa20 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:12.410424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:12.422440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:12.508157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c00d-1669-4161-acd4-1ef39d1592ba None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:12.539888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:12.540552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:12.540833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:12.541108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] PATH_INFO: `/v3/projects/f617faf352cd4576bb066475dd70fa20/users/61f9855b1e674ce0b6234b30a6e8c187/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:12.543217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] RBAC: Authorizing `identity:create_grant(project_id=f617faf352cd4576bb066475dd70fa20, user_id=61f9855b1e674ce0b6234b30a6e8c187, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:12.545178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66117f65-0b0c-4f1b-b55c-991ebcf0419a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:12.648491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 599/1153] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:49:12 2022] PUT /identity/v3/projects/f617faf352cd4576bb066475dd70fa20/users/61f9855b1e674ce0b6234b30a6e8c187/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:49:14.503909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a8bc-e8ea-47e9-a516-29cea1c56a47 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:14.504587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a8bc-e8ea-47e9-a516-29cea1c56a47 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:14.504587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a8bc-e8ea-47e9-a516-29cea1c56a47 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:14.505800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 555/1154] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:14.510920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:14.511295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:14.511474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:14.529756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:14.530934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:14.592361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:14.597821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 600/1155] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:14.866988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982a8bc-e8ea-47e9-a516-29cea1c56a47 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:14.969704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982a8bc-e8ea-47e9-a516-29cea1c56a47 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.003667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.004326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.004606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.004884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.015447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:15.020220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:15.021337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:49:15.022568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 556/1156] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:15.027373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:15.124850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e1df0-bdf1-4fa8-8589-c92521f14fc4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.157304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.157905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.158185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.158457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.159115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:15.159418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:15.160811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:15.168895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 601/1157] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:15.175260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:15.205563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:15.273381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4715-6b89-41bf-812e-46de3e6e59ed None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.305476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.306084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.306359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.306627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] PATH_INFO: `/v3/users/61f9855b1e674ce0b6234b30a6e8c187` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.307718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] RBAC: Authorizing `identity:get_user(user_id=61f9855b1e674ce0b6234b30a6e8c187)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:15.309158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:15.311186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 557/1158] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:15 2022] GET /identity/v3/users/61f9855b1e674ce0b6234b30a6e8c187 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:15.317245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:15.374963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:15.414199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bc05-fa3e-4ab7-a3e7-01dd15fd94af None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.447220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.447820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.448097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.448394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.512665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:15.514637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:15.569004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:15.610311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 602/1159] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:15.616397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:15.708781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:15.712320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588f8f-3404-48ca-951a-4c6938446a11 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.744425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.745085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.745361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.745631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] PATH_INFO: `/v3/projects/f617faf352cd4576bb066475dd70fa20` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.746741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] RBAC: Authorizing `identity:get_project(project_id=f617faf352cd4576bb066475dd70fa20)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:15.748283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437baa27-ce67-49cd-988d-daf605b5ed52 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:15.749867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 558/1160] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:15 2022] GET /identity/v3/projects/f617faf352cd4576bb066475dd70fa20 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:15.755987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:15.851607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439da0a-ac87-4abb-a00b-99800ea88090 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:15.883675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:15.884282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:15.884556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:15.884822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:15.885526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=f617faf352cd4576bb066475dd70fa20, user.id=61f9855b1e674ce0b6234b30a6e8c187) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:15.885830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:15.887228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7541c-3298-4034-8058-4545621fec1f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:15.893226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 603/1161] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:49:15 2022] GET /identity/v3/role_assignments?user.id=61f9855b1e674ce0b6234b30a6e8c187&scope.project.id=f617faf352cd4576bb066475dd70fa20&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:17.823573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901968-10dc-49df-b11b-336904840137 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:17.824177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901968-10dc-49df-b11b-336904840137 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:17.824177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901968-10dc-49df-b11b-336904840137 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:17.825485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 559/1162] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:17.830765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:17.831206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:17.831302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:17.849866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:17.851028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:17.868407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:17.918459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 604/1163] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:18.204062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6901968-10dc-49df-b11b-336904840137 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:18.302012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6901968-10dc-49df-b11b-336904840137 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:18.335085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:18.335674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:18.335953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:18.336251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:18.367869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:18.401164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:18.402957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:18.497360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 560/1164] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:18.503765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:18.598702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cfbfa-6bb4-4ef5-b868-b92f92f19e8d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:18.632488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:18.633117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:18.633398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:18.633669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:18.640782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:18.642206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1077e777-27af-479f-9180-e6f2e345ef73 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:18.653200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:18.654574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:18.655580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 605/1165] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:49:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:18.660701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:18.724425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:18.758993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50ac67-50ca-4dba-af26-06e0378ab2ac None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:18.791193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:18.791780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:18.792323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:18.792607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:18.793239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:18.793541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:18.795680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c97d8-ed9a-4fd0-99e2-70ca77f57d18 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:18.802599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 561/1166] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Aug 25 02:49:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:20.689391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09df4-5e2a-409c-9acb-f07a15115c61 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:20.689920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09df4-5e2a-409c-9acb-f07a15115c61 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:20.689920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09df4-5e2a-409c-9acb-f07a15115c61 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:20.691295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 606/1167] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:20.696623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73924-bfdb-492e-8438-7eb459973d0a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:20.697017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73924-bfdb-492e-8438-7eb459973d0a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:20.697178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73924-bfdb-492e-8438-7eb459973d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:20.715777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-43b73924-bfdb-492e-8438-7eb459973d0a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:20.716992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-43b73924-bfdb-492e-8438-7eb459973d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:20.782691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 562/1168] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:21.060792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e09df4-5e2a-409c-9acb-f07a15115c61 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:21.077087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:21.159982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e09df4-5e2a-409c-9acb-f07a15115c61 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:21.195103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:21.195696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:21.195973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:21.196272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:21.196926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:21.198273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471de57d-be40-4b40-8b6b-aa18478d59be None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:21.291507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 607/1169] 10.222.0.75 () {60 vars in 1138 bytes} [Thu Aug 25 02:49:21 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:49:23.240419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454a41-8389-48d4-abaf-2ec01cf51800 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:23.240937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454a41-8389-48d4-abaf-2ec01cf51800 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:23.240937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34454a41-8389-48d4-abaf-2ec01cf51800 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:23.242170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 563/1170] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:23.247365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:23.247793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:23.247925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:23.267006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:23.268253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:23.333334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 608/1171] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:23.605720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34454a41-8389-48d4-abaf-2ec01cf51800 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:23.663853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:23.705860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34454a41-8389-48d4-abaf-2ec01cf51800 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:23.737888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:23.738533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:23.738832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:23.739100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] PATH_INFO: `/v3/domains/b9be90901d23476b919a3e31aafa9b91` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:23.750056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9be90901d23476b919a3e31aafa9b91)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:23.751489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5b4a9-53c4-442c-9c80-fbaee771c000 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:23.753069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 564/1172] 10.222.0.75 () {56 vars in 1183 bytes} [Thu Aug 25 02:49:23 2022] GET /identity/v3/domains/b9be90901d23476b919a3e31aafa9b91 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:23.758801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:23.761983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:23.855246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9820f7-ca76-43ff-a8cf-a46a16422bae None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:23.888699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:23.889318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:23.889597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:23.889868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:23.890512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:23.891923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b26338-cc71-4cd9-82f8-24fbb9088074 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:23.987556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 609/1173] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:49:23 2022] POST /identity/v3/projects => generated 361 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:49:25.916210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034762f-285a-469b-b3d6-0524784832fd None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:25.916762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034762f-285a-469b-b3d6-0524784832fd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:25.916762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034762f-285a-469b-b3d6-0524784832fd None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:25.918343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 565/1174] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:25.923732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:25.924186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:25.924302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:25.943325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:25.944569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:25.993195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:26.010010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 610/1175] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:49:26.291433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034762f-285a-469b-b3d6-0524784832fd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:26.393440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034762f-285a-469b-b3d6-0524784832fd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:26.413187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:26.427134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:26.427763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:26.429081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:26.429081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] PATH_INFO: `/v3/domains/b9be90901d23476b919a3e31aafa9b91` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:26.429420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9be90901d23476b919a3e31aafa9b91)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:26.430854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629966-5254-4328-ad39-6d316fcc3004 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:26.432442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 566/1176] 10.222.0.75 () {56 vars in 1183 bytes} [Thu Aug 25 02:49:26 2022] GET /identity/v3/domains/b9be90901d23476b919a3e31aafa9b91 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:26.438255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:26.534771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe400a32-8863-4f2f-85bf-b02daec50316 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:26.570732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:26.571273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:26.571555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:26.571828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:26.572533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:26.573954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371c6e4-9ebd-431a-8ae3-45ac327c2d3f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:26.688772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 611/1177] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:49:26 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:49:28.618086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501868-4593-4e83-96be-c69ca2844aa8 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:28.618648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501868-4593-4e83-96be-c69ca2844aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:28.618648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501868-4593-4e83-96be-c69ca2844aa8 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:28.619948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 567/1178] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:28.625793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:28.626244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:28.626347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:28.644046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:28.645244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:28.679268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:28.709640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 612/1179] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:28.978673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1501868-4593-4e83-96be-c69ca2844aa8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.078513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1501868-4593-4e83-96be-c69ca2844aa8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:29.111436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:29.112053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:29.112360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:29.112638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:29.122359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:29.127126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:29.128320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:49:29.129665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 568/1180] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:29.134117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.230352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55617-233d-4a78-a8ce-bc78742ea96b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:29.262185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:29.262776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:29.263052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:29.263322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:29.264795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:29.265028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:29.266371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:29.274016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 613/1181] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:29.280791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.330261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:29.378348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5abc-f4bb-4680-a7a7-a3a9ea2b7fef None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:29.411300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:29.411917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:29.412220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:29.412493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] PATH_INFO: `/v3/users/3c8ad22ac36642759f78357de78e5bed` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:29.425417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8ad22ac36642759f78357de78e5bed)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:29.426876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:29.428668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 569/1182] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:29 2022] GET /identity/v3/users/3c8ad22ac36642759f78357de78e5bed => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:29.434529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.469650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:29.532895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793dcd3-1da4-4cfc-a60d-8ddfc439ce94 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:29.565433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:29.566046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:29.566307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:29.566579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:29.632226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:29.633657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:29.665748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:29.729579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 614/1183] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:29.735541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.830985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9109c-f97b-4655-ac6c-d342a732b944 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:29.837939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:29.863292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:29.863899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:29.864205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:29.864481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] PATH_INFO: `/v3/projects/4196a97a0be54dc683c0f931603f0256` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:29.865610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] RBAC: Authorizing `identity:get_project(project_id=4196a97a0be54dc683c0f931603f0256)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:29.867033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c8312-7d2d-49d3-bd08-26027ea623f8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:29.868659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 570/1184] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:29 2022] GET /identity/v3/projects/4196a97a0be54dc683c0f931603f0256 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:29.875343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:29.971303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbf7b-5567-41fb-8f4f-81c7aecb38c8 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:30.004564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:30.005154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:30.005429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:30.005696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:30.006405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=4196a97a0be54dc683c0f931603f0256, user.id=3c8ad22ac36642759f78357de78e5bed) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:30.006707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:30.008114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f024cc-4290-430c-bfdb-34e173379164 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:30.009816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:30.013693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 615/1185] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:49:29 2022] GET /identity/v3/role_assignments?user.id=3c8ad22ac36642759f78357de78e5bed&scope.project.id=4196a97a0be54dc683c0f931603f0256&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:31.882376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ef22-600d-4597-b046-36c97e6c2da2 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:31.882926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ef22-600d-4597-b046-36c97e6c2da2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:31.882926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ef22-600d-4597-b046-36c97e6c2da2 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:31.884286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 571/1186] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:31.889934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:31.890365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:31.890464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:31.908046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:31.909231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:31.974311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 616/1187] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:32.249087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ef22-600d-4597-b046-36c97e6c2da2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:32.346059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ef22-600d-4597-b046-36c97e6c2da2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:32.381425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:32.382037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:32.382312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:32.382580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:32.392229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:32.396865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:32.397931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:49:32.398801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 572/1188] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:32.404007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:32.504660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a2e09-734a-44fc-814b-f7329bbfe33a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:32.530922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:32.538260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:32.538856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:32.539132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:32.539400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:32.540049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:32.540380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:32.541991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:32.550251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 617/1189] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:32.556248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:32.590267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:32.653385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dab2-2e01-486c-9d61-76784bace167 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:32.687066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:32.687674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:32.687955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:32.688250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] PATH_INFO: `/v3/users/3c8ad22ac36642759f78357de78e5bed` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:32.689342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8ad22ac36642759f78357de78e5bed)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:32.690793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:32.692360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 573/1190] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:32 2022] GET /identity/v3/users/3c8ad22ac36642759f78357de78e5bed => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:32.698032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:32.794468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b4a28-e3e3-42db-a04c-ff56b3891e5c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:32.826693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:32.827308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:32.827582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:32.827851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] PATH_INFO: `/v3/projects/4196a97a0be54dc683c0f931603f0256` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:32.828966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] RBAC: Authorizing `identity:get_project(project_id=4196a97a0be54dc683c0f931603f0256)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:32.830379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a432f-3000-42b0-9375-da52f129793c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:32.831973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 618/1191] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:32 2022] GET /identity/v3/projects/4196a97a0be54dc683c0f931603f0256 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:32.837957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:32.934768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee29cb0-f0aa-4778-bcc3-7229952150f1 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:32.936341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:32.969314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:32.969962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:32.970241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:32.970511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] PATH_INFO: `/v3/projects/4196a97a0be54dc683c0f931603f0256/users/3c8ad22ac36642759f78357de78e5bed/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:32.972410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] RBAC: Authorizing `identity:create_grant(project_id=4196a97a0be54dc683c0f931603f0256, user_id=3c8ad22ac36642759f78357de78e5bed, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:32.973832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633188-e9e0-469c-8a45-82c9f2282052 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:33.064968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 574/1192] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:49:32 2022] PUT /identity/v3/projects/4196a97a0be54dc683c0f931603f0256/users/3c8ad22ac36642759f78357de78e5bed/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:49:34.944951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3338fb-187b-402b-a9f0-2b637992953d None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:34.945390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3338fb-187b-402b-a9f0-2b637992953d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:34.945493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3338fb-187b-402b-a9f0-2b637992953d None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:34.946843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 619/1193] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:49:34.952103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:34.952576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:34.952669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:34.970830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:49:34.972011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:49:35.037964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 575/1194] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:49:35.319520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3338fb-187b-402b-a9f0-2b637992953d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:35.379162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:35.421372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3338fb-187b-402b-a9f0-2b637992953d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:35.456798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:35.457433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:35.457718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:35.457992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:35.469265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:49:35.474034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:49:35.475149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:49:35.476060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 620/1195] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:49:35.481214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:35.567168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:35.575507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4d8c6-411f-4c26-9646-7956100496c1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:35.607943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:35.608557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:35.608834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:35.609103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:35.609752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:35.610052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:35.611412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:35.619395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 576/1196] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:35.625403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:35.726650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bd7b-713d-4285-90e2-359b0a80e049 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:35.744802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:35.761367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] PATH_INFO: `/v3/users/3c8ad22ac36642759f78357de78e5bed` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] RBAC: Authorizing `identity:get_user(user_id=3c8ad22ac36642759f78357de78e5bed)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:35.778037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 621/1197] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Aug 25 02:49:35 2022] GET /identity/v3/users/3c8ad22ac36642759f78357de78e5bed => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:35.778566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:35.871145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d06ad-49e8-4d60-a84c-edd5216e80a2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:35.904006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:35.905184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:35.905463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:35.905733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:35.929645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:35.973743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:35.975222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:36.068752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 577/1198] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:49:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:49:36.074674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:36.178823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2801-816d-48d8-b2a5-25c3ae82b10d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:36.210962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:36.211579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:36.211857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:36.212128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] PATH_INFO: `/v3/projects/4196a97a0be54dc683c0f931603f0256` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:36.213248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] RBAC: Authorizing `identity:get_project(project_id=4196a97a0be54dc683c0f931603f0256)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:36.214671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b043463-48ca-4f02-be5c-deb2bce11716 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:36.216305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 622/1199] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Aug 25 02:49:36 2022] GET /identity/v3/projects/4196a97a0be54dc683c0f931603f0256 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:36.223304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:49:36.277515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:36.321078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd341a-10b0-457a-9a71-e12a641dda62 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:49:36.353355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:49:36.353948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:49:36.354224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:49:36.354495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:49:36.355207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=4196a97a0be54dc683c0f931603f0256, user.id=3c8ad22ac36642759f78357de78e5bed) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:49:36.355514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:49:36.356963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968fa12-3d9d-4dc7-bc19-34d66b1e9a9f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:49:36.362584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 578/1200] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:49:36 2022] GET /identity/v3/role_assignments?user.id=3c8ad22ac36642759f78357de78e5bed&scope.project.id=4196a97a0be54dc683c0f931603f0256&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:49:46.207016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=73 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:49:46.360074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=41 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:50.872862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d67a9-df65-42ef-83a4-3f9f66bec1b4 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:50.873458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d67a9-df65-42ef-83a4-3f9f66bec1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:50.873458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d67a9-df65-42ef-83a4-3f9f66bec1b4 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:50.874766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 623/1201] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:50:50.879788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:50.880215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:50.881067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:50.898561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:50:50.900183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:50:50.966538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 579/1202] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:50:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:50:51.234380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d67a9-df65-42ef-83a4-3f9f66bec1b4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:51.337146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d67a9-df65-42ef-83a4-3f9f66bec1b4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:51.370021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:51.373109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:51.373109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:51.373109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:51.382544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:51.383972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:50:51.398808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:51.405693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:50:51.406701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:50:51.406701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:51.406701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:50:51.406701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 624/1203] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:51.406966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:51.505334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1143c-ca11-4f57-88eb-4fa6e9c85886 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:51.536112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:51.542549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:51.543160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:51.543440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:51.543725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:51.544402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:51.546166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:51.546166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dce005-7872-45e9-8314-09c5257a62ce None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:51.557232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 580/1204] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:51.563122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:51.615519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:51.663698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750626fe-c442-4825-8d18-47beb8fe280d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:51.696781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:51.697370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:51.697647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:51.697913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:51.698536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:51.699942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde18ad7-6705-46f6-84bb-12aa0c43329a None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:51.815449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 625/1205] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Aug 25 02:50:51 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:50:53.779688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a972-5112-4201-b2b5-5fbaedcd4039 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:53.780264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a972-5112-4201-b2b5-5fbaedcd4039 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:53.780264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281a972-5112-4201-b2b5-5fbaedcd4039 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:53.781676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 581/1206] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:50:53.787683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:53.789701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:53.789701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:53.806277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:50:53.807431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:50:53.871700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 626/1207] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:50:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:50:54.141418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a972-5112-4201-b2b5-5fbaedcd4039 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:54.240022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281a972-5112-4201-b2b5-5fbaedcd4039 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:54.277392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:54.278013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:54.278291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:54.278557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:54.289001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:50:54.293934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:54.295093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:54.296177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 582/1208] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:54.303881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:54.345748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:54.406647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb2d6-09e4-4c70-8990-600970a7b958 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:54.441306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:54.441908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:54.442192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:54.442463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:54.444768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:54.445074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:54.446465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:54.454266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 627/1209] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:54.460238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:54.512422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:54.560764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da95b9-3b3b-41b9-87bb-238151a45f1e None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:54.594382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:54.595004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:54.595284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:54.595558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:54.606124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:54.607570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:50:54.622292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:54.623256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:50:54.624275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:50:54.624275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:54.624275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:54.624275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 583/1210] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:54.629846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:54.734850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:54.737673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871194ea-328e-45cf-85d3-e1c1c78a9f98 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:54.770515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:54.771507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:54.771799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:54.772072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:54.772735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:54.773043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:54.774465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:54.785765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 628/1211] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:54.792544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:54.890955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76438f83-d2e3-4bb8-8b09-769a5ea0a4e6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:54.892482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:54.925753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:54.926383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:54.926662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:54.926964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:54.935166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:54.936984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:50:54.948562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:54.949838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:54.951257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 584/1212] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:54.955497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:55.059566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9a860-6906-4f75-83cc-9fa54b4e1a84 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:55.093609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:55.094217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:55.094492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:55.094759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:55.095595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:55.095902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:55.097358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:55.115104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 629/1213] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:50:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:55.121212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:55.221334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e52bc1-94c8-49cb-9097-d4ea43884709 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:55.254985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:55.256209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:55.256567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:55.256946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:55.285185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:55.324632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:55.326004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:55.420993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 585/1214] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:50:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:50:55.426879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:55.447015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:55.526427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad7368-39fd-4ca4-ae38-c1fa9935ba64 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:55.561759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:55.562372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:55.562651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:55.562918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:55.574862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:55.576348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:50:55.591909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:55.593089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:50:55.594430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:50:55.594430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:55.594430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:50:55.594430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 630/1215] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:55.600169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:55.660654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:55.699106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012590fa-1fe5-47b2-8aaf-66d4cb0acd39 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:55.731881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:55.732525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:55.732807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:55.735019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:55.735751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:55.736070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:55.737506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:55.749561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 586/1216] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:55.755270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:55.824294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:55.855478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c0e8-0730-4813-b386-fe7f9ce91df9 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:55.887350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:55.887970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:55.888277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:55.888553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:55.895078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:55.897251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:50:55.908368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:55.909360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:50:55.910269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 631/1217] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:55.916527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:56.016457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ecc8-b8a9-487c-8759-af815b0951ea None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:56.040186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:56.051284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:56.051880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:56.052817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:56.053092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:56.053780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:56.054087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:56.055510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6059bf2-edf4-400f-a6f3-827f499fa3c0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:56.066396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 587/1218] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:56.072777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:56.180013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d03d5-123a-40e5-ac09-9cb7cc9aad96 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:56.207412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:56.213639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:56.214235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:56.214513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:56.214787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:56.215503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=8912c5edc2964df1afb96fd23f02e299) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:56.215809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:56.217325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34f7c-4e80-4110-b36c-88f2224d32cf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:56.222505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 632/1219] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:50:56 2022] GET /identity/v3/role_assignments?user.id=8912c5edc2964df1afb96fd23f02e299&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:58.129004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19783d2-811b-465b-a89d-98b24782e5c1 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:58.129569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19783d2-811b-465b-a89d-98b24782e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:58.129569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19783d2-811b-465b-a89d-98b24782e5c1 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:58.131944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 588/1220] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:50:58.137401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:58.137796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:58.137939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:58.155842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:50:58.161296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:50:58.223354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 633/1221] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:50:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:50:58.497277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19783d2-811b-465b-a89d-98b24782e5c1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:58.599493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19783d2-811b-465b-a89d-98b24782e5c1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:58.633987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:58.634616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:58.634894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:58.635167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:58.645550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:50:58.651267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:58.652268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:58.653248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 589/1222] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:58.658488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:58.761124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dfcb1-ec90-40a9-8d98-fca43f7f8678 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:58.786726 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:58.797558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:58.799928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:58.799928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:58.799928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:58.799928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:58.799928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:58.801337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:58.809732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 634/1223] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:58.815666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:58.859341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:58.913627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece35a1b-260d-4fe5-a28f-674184baf2ac None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:58.947701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:58.948367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:58.948646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:58.948918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:58.958681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:58.960208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:50:58.976452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:58.977730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:50:58.978958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:50:58.978958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:58.978958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:58.978958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 590/1224] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:58.983579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.087068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d193795-a58d-4692-bec6-81e99576b7fd None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.120286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.120881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.121157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.121426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.122053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:59.122351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.123743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.135879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 635/1225] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:59.141769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.234907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:59.242164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329a3be-f89e-4d9f-a9f2-dc04906fef59 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.274691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.275311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.275586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.275853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.282722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.287152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:50:59.296050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:59.297207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:59.298400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 591/1226] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:59.303686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.352102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:59.404177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423197e-559c-4dfc-b3f0-813b3b9d95e2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.438827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.439425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.439708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.439985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.440736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:59.441047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.442561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.459680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 636/1227] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:59.465420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.564923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fee7-3db8-4509-9afa-5a82bb28b9b7 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.597729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.598354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.598635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.599533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.609824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.611222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:50:59.625111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:59.626282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:50:59.627230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:50:59.627230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:50:59.627230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:59.627230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 592/1228] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:59.632206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.731323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ab20b-063f-4fb7-a00f-a547ba11924d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.733696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:59.764274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.764875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.765198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.765467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.766107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:50:59.766411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.767763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.779024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 637/1229] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:50:59.784791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:50:59.795027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:50:59.882105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bda8-1122-4ef5-a04c-e0d401dc7fc2 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:50:59.915596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:50:59.916257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:50:59.916541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:50:59.916817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:50:59.923560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:50:59.925056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:50:59.937570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:50:59.938666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:50:59.939961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 593/1230] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:50:59.944919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:00.046696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef278513-6113-43cd-8528-986ca250b6c2 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:00.080165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:00.080782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:00.081062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:00.081336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:00.082024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:00.082373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:00.084625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63d117-d073-456d-8ce1-a005aa57dde3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:00.096365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 638/1231] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:00.102594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:00.167998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:00.201971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b43e-c2ee-4449-bd28-97352a71332c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:00.235218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:00.235881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:00.236178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:00.236456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/8912c5edc2964df1afb96fd23f02e299/roles/241d7bddb40c406492616417acb4d1e5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:00.251508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=8912c5edc2964df1afb96fd23f02e299, role_id=241d7bddb40c406492616417acb4d1e5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:00.252957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77cb84-e802-4ffa-9dbb-2e061e58e4d0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:00.315145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 594/1232] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:51:00 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/8912c5edc2964df1afb96fd23f02e299/roles/241d7bddb40c406492616417acb4d1e5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:51:02.209745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f28e1-ebc1-4893-979b-c3dd4c7a9a58 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:02.210292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f28e1-ebc1-4893-979b-c3dd4c7a9a58 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:02.210292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f28e1-ebc1-4893-979b-c3dd4c7a9a58 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:02.211626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 639/1233] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:02.216890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:02.217332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:02.217441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:02.235316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:02.236515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:02.303637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 595/1234] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:02.580925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f28e1-ebc1-4893-979b-c3dd4c7a9a58 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:02.606610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:02.681986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f28e1-ebc1-4893-979b-c3dd4c7a9a58 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:02.715683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:02.716326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:02.716606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:02.716876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:02.727012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:51:02.731805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:02.733001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:02.733925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 640/1235] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Aug 25 02:51:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:02.739063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:02.786815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:02.839338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b743efa-4d7d-4f88-ad1e-c0c618a70977 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:02.873165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:02.873813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:02.874094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:02.874370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:02.875031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:02.875340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:02.876747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:02.885392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 596/1236] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Aug 25 02:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:02.891314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:02.977522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:02.993833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb421d-608f-4216-94ec-1f9facccec0f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.034444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.035069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.035350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.035616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.046108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.047535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:03.062089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:03.063196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:03.064135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:03.064135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:03.064135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:03.064135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 641/1237] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:03.070247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:03.171561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d739-ee11-4601-8a91-bcc8dc8fed2a None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.180322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:03.204511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.205110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.205392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.205664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.206304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:03.206609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.208628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:03.220294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 597/1238] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:03.226783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:03.326195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e8568-f53c-4aff-9e8a-5ba83a4dfa53 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.360002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.360653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.360931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.361201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.363776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:03.368661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.370076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:51:03.380810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:03.382042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:03.383409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 642/1239] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:03.387786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:03.488022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd204a3c-eed9-4200-911f-24bcc7ed6480 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.521789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.522377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.524630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.524630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.524630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:03.524630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.525485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:03.543473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 598/1240] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:51:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:03.549260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:03.649379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59a37-e4a4-4d5d-afa3-a54529c984d0 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.682508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.683101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.683377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.683650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.744190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:03.750383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.751814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:03.849404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 643/1241] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:51:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:51:03.855661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:03.883194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:03.953196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b922d-2641-4aea-b697-bc4536e2ce1d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:03.985741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:03.986359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:03.986633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:03.986899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:03.997425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:03.998798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:04.012784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:04.013893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:04.014999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:04.014999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:04.014999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:04.014999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 599/1242] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:04.019611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:04.117608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:04.123716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a096a-ec94-44c4-a4c7-2bf73d922193 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:04.158582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:04.159213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:04.159496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:04.159773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:04.160462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:04.160767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:04.162906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:04.174163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 644/1243] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:04.180363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:04.260546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:04.281779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0ba3-ad66-4cc6-83ec-5a82d2175c04 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:04.315335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:04.315980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:04.316298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:04.316567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:04.324229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:04.325685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:51:04.337092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:04.338692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:04.339806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 600/1244] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:51:04.344107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:04.442203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df650-acd5-4e87-a0ae-6659ccde05d6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:04.475450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:04.476041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:04.476353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:04.476623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:04.477302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:04.477606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:04.479017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c765-6f81-440f-82b6-64835808fff2 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:04.491380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 645/1245] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:04.497602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:04.597662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93464e-7a56-4037-b91c-62c687954b71 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:04.631724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:04.632349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:04.632628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:04.632897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:04.633618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] RBAC: Adding query filter params (role.id=241d7bddb40c406492616417acb4d1e5, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=8912c5edc2964df1afb96fd23f02e299) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:04.633928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:04.635355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d56f22-31fe-4cc2-b27d-b3da7959706d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:04.641198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 601/1246] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:51:04 2022] GET /identity/v3/role_assignments?user.id=8912c5edc2964df1afb96fd23f02e299&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=241d7bddb40c406492616417acb4d1e5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:06.639116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf037-6672-4e6f-8465-91559faf562d None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:06.639705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf037-6672-4e6f-8465-91559faf562d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:06.639705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbf037-6672-4e6f-8465-91559faf562d None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:06.641133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 646/1247] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:06.646893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:06.647293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:06.647441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:06.653754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:06.666508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:06.667686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:06.732880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 602/1248] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 25 02:51:07.002607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbf037-6672-4e6f-8465-91559faf562d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.022018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:07.112589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbf037-6672-4e6f-8465-91559faf562d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.146465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.147081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.147356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.147620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.157235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:51:07.162116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:07.163220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:07.164533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 647/1249] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:07.169422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.266260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ddc98-4b37-44aa-8592-79aa2f6c98da None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.295125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:07.299952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.300575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.300852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.301121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.301772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:07.302073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:07.303953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:07.311757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 603/1250] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:07.317617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.395827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:07.419188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c16cd-72cb-4a1b-b667-22628182c23a None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.452212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.453073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.453366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.454002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.465272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:07.466713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:07.481318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:07.482445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:07.483494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:07.483494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:07.483494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:07.483494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 648/1251] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:07.488415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.586593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd834a1-6781-481b-995e-db22bd789e0c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.619278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.619880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.620171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.620467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.621103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:07.621405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:07.622765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:07.635096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 604/1252] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:07.641039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.740519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb074fe-b91c-484d-9ebd-8751cacdde95 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.766322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:07.774695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.775324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.775599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.775868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.786195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:07.787715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:51:07.798902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:07.800044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:07.801033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 649/1253] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:07.806896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:07.862241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:07.909519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2910d-3aa6-40ed-a8ad-891f20f895fa None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:07.944719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:07.945314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:07.945588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:07.945855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:07.946566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:07.946876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:07.948305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:07.965867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 605/1254] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:07.971758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.073754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6763c-9848-4cae-be10-facf3a1061ba None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:08.106873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:08.107468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:08.107744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:08.108012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:08.165671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:08.177688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:08.179800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:08.276287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 650/1255] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:51:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:51:08.281994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.380234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969bd2-4197-4bb8-8000-f9a80b21f96f None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:08.392917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:08.413785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:08.414662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:08.414937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:08.415208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:08.425459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:08.426847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:08.441991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:08.444464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:08.446747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:08.446747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:08.446747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:08.446747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 606/1256] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:08.449167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.536336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:08.549176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a74423-73a8-4fc8-a046-26e739bae687 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:08.583056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:08.583656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:08.583940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:08.584243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:08.585505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:08.585901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:08.587265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:08.598346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 651/1257] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:08.604517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.704634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50993-47cc-41c2-b7ee-daf3b18bf8da None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:08.737321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:08.737942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:08.738219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:08.738490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:08.747475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:08.749091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:51:08.761641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:08.762752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:08.763822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 607/1258] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:08.769315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.871022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04634fae-8e87-4130-b778-3018c14c266c None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:08.904253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:08.904848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:08.905122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:08.905390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:08.906065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:08.906368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:08.907856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80191d62-4b9f-4aeb-8758-4e01586af48d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:08.920890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 652/1259] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:08.927124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:08.953667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:09.028395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1320f9f-b9c1-4cd2-97fc-72ae4cc554d9 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:09.062138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:09.062808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:09.063108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:09.063379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:09.064094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=8912c5edc2964df1afb96fd23f02e299) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:09.064430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:09.065859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b03a0-e6b8-4ce4-843b-0ef76b369f17 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:09.071048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 608/1260] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:51:08 2022] GET /identity/v3/role_assignments?user.id=8912c5edc2964df1afb96fd23f02e299&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:10.957872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9a94-37db-40e3-be77-42fcd0666b6e None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:10.958321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9a94-37db-40e3-be77-42fcd0666b6e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:10.958436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9a94-37db-40e3-be77-42fcd0666b6e None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:10.960359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 653/1261] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:10.966014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:10.966420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:10.966596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:10.984878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:10.986057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:11.050870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 609/1262] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:11.331933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9a94-37db-40e3-be77-42fcd0666b6e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:11.342955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:11.432104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9a94-37db-40e3-be77-42fcd0666b6e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:11.465905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:11.466527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:11.466809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:11.467086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:11.476972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:51:11.481674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:11.482778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:11.483757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 654/1263] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:11.489045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:11.509165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:11.589145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a93791-144b-41fb-8f35-630315fb45ca None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:11.622532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:11.623127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:11.623403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:11.623672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:11.624351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:11.624650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:11.626007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:11.634671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 610/1264] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:11.640577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:11.707264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:11.739389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb768-d4fa-40d2-88c2-370f160a71e5 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:11.773362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:11.774066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:11.774374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:11.774633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:11.785512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:11.786942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:11.801185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:11.802328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:11.804413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:11.804413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:11.804413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:11.804413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 655/1265] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:11.808857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:11.897022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:11.917055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6aad-5630-49b3-81c9-a789d695a1e6 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:11.951745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:11.952388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:11.952667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:11.952968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:11.953603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:11.953910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:11.955266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:11.966586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 611/1266] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:11.972945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.073115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbbbb3-93b3-4fdb-831a-fb267bd35f4d None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.092527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:12.107678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.108342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.108618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.108883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.117058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.118644 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:51:12.129503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:12.135775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:12.139142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 656/1267] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:12.139860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.241942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922906b9-9137-45d4-896a-45d2251c232c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.274355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.274952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.275229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.275499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.276247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:12.286126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.286126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.286753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:12.295259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 612/1268] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:51:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:12.301167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.402734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028722-3a11-4c47-acb0-7ef4301d4ce1 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.436645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.437266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.437544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.437815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.448461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.449888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:12.464937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:12.466151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:12.467410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:12.467410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:12.467410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:12.470965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 657/1269] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:12.472107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.571419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8720ec-a60d-43c0-85ef-d736f9e79595 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.605884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.612836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.621087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 613/1270] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:12.626785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.659138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:12.727165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672df77-fb08-4af4-88a7-42b7b8ce9119 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.760193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.760849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.761134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.761408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.767964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.769417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:51:12.781818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:12.782881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:12.783993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 658/1271] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:12.789792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:12.848313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:12.889125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e707e3-6347-4648-bb9e-ddf275a32fed None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:12.922152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:12.922753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:12.923034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:12.923307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:12.923994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:12.924327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:12.925749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9c870-cf30-4545-8a02-69e866a0c669 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:12.937880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 614/1272] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:51:12.944533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:13.038093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:13.046901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab80d25-c6f9-49f5-9a7f-72008cd9adaa None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:13.080325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:13.080965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:13.081238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:13.081505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] PATH_INFO: `/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/8912c5edc2964df1afb96fd23f02e299/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:13.083355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] RBAC: Authorizing `identity:create_grant(project_id=932be52c2fb94ef7bad55913e4e24363, user_id=8912c5edc2964df1afb96fd23f02e299, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:13.085462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8dec2-7839-4a78-aab8-82913cb6eaea None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:13.196835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 659/1273] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Aug 25 02:51:12 2022] PUT /identity/v3/projects/932be52c2fb94ef7bad55913e4e24363/users/8912c5edc2964df1afb96fd23f02e299/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 02:51:15.098792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672f8-a08b-4c67-b29c-75364f1d1e10 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:15.099318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672f8-a08b-4c67-b29c-75364f1d1e10 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:15.099438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672f8-a08b-4c67-b29c-75364f1d1e10 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:15.100754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 615/1274] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:15.105981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:15.106384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:15.106558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:15.125233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:15.126420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:15.193307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 660/1275] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:15.473656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c672f8-a08b-4c67-b29c-75364f1d1e10 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:15.576934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c672f8-a08b-4c67-b29c-75364f1d1e10 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:15.594588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:15.610970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:15.611586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:15.611866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:15.612137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:15.622392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 25 02:51:15.627087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:15.628231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:15.634199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 616/1276] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Aug 25 02:51:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:15.635419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:15.715990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:15.734048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b378f5d-b21b-47a4-8d90-27a474c55cb3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:15.766797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:15.767398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:15.767678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:15.767972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:15.768667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:15.768967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:15.770311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:15.778105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 661/1277] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Aug 25 02:51:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:15.783799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:15.885642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f704-f54f-45e3-822b-f751460ce386 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:15.920455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:15.921114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:15.921420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:15.921697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:15.932716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:15.934180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:15.951517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:15.952695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:15.953614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:15.953614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:15.953614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:15.953614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 617/1278] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:15.958846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:16.058547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc33f-f2b1-408d-b409-3ceebc460cf6 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:16.091479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:16.092082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:16.092390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:16.092665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:16.093307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:16.093613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:16.094971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:16.107192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 662/1279] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:16.112948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:16.143027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:16.211007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a0df5-a373-4251-99d2-d4c6f0fb2ec0 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:16.244853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:16.245475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:16.245754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:16.246026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:16.252786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:16.254287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 25 02:51:16.266165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:16.267270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:16.268450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 618/1280] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:16.273947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:16.277805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:16.373907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fde2-073d-4694-824d-6d77eef5d158 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:16.407326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:16.407920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:16.408222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:16.408494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:16.409205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:16.409504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:16.410906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:16.429714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 663/1281] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Aug 25 02:51:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:16.435391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:16.525907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:16.535177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2785601-5133-4150-8da7-3678b1c5ef8d None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:16.575155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:16.575842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:16.576214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:16.576534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:16.643934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:16.645347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:16.731825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:16.742666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 619/1282] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Aug 25 02:51:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 25 02:51:16.748607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:16.811742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:16.849484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99189e9-5915-4004-9f2c-94db39b4076f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:16.882398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:16.883008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:16.883287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:16.883554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:16.893218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:16.894761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 25 02:51:16.909668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:16.916768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 25 02:51:16.917719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 25 02:51:16.917719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 25 02:51:16.917719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:16.917719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 664/1283] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 02:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:16.917972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:17.015422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3ff7-45a3-43e8-a477-c4461075b6a8 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:17.049990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:17.050584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:17.050859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:17.051127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:17.051755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:17.052056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:17.053498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:17.064760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 620/1284] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Aug 25 02:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:17.071090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:17.173158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d26f-83ac-4ac9-9657-d32f7b91f254 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:17.186646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:17.206478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:17.207105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:17.211408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:17.211408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:17.214677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:17.216094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 25 02:51:17.227210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:17.228267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: ERROR keystone.server.flask.application Aug 25 02:51:17.229350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 665/1285] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Aug 25 02:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:17.234203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:17.284328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:17.332325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9dd6f-9fe8-4402-95d0-c7f0a52101ec None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:17.366302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:17.367008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:17.367224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:17.367485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:17.368204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:17.379983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:17.379983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae41f1d-68e9-4889-bba4-231f00c9a64e None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:17.382845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 621/1286] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Aug 25 02:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:17.389033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:17.487662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d966-04cd-4d0d-a812-f1941bf8584f None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:17.520414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:17.521007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:17.521289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:17.521566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:17.522846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] RBAC: Adding query filter params (role.id=a6f0106bea874efe895b3a8ed472a48f, scope.project.id=932be52c2fb94ef7bad55913e4e24363, user.id=8912c5edc2964df1afb96fd23f02e299) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:17.523294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:17.524748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55830e63-57dc-4ce4-9ad9-4b7303d004f5 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:17.530413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 666/1287] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Aug 25 02:51:17 2022] GET /identity/v3/role_assignments?user.id=8912c5edc2964df1afb96fd23f02e299&scope.project.id=932be52c2fb94ef7bad55913e4e24363&role.id=a6f0106bea874efe895b3a8ed472a48f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:19.460581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f9ed-208e-40da-b31c-fee2c63d5e9c None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:19.461165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f9ed-208e-40da-b31c-fee2c63d5e9c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:19.461165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f9ed-208e-40da-b31c-fee2c63d5e9c None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:19.462565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 622/1288] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:19.467936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:19.468393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:19.468503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:19.486606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:19.487781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:19.517582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:19.553897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 667/1289] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:19.819757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f9ed-208e-40da-b31c-fee2c63d5e9c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:19.920873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f9ed-208e-40da-b31c-fee2c63d5e9c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:19.933239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:19.960928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:19.961552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:19.961828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:19.962097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:19.962673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:19.964025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:19.972793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:19.979286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:19.980373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 623/1290] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:51:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:19.981378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:20.087061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033a9e-6936-4b83-a67b-d6440cbbfb55 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:20.124597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:20.125231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:20.125508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:20.125780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:20.126534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:20.126740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:20.128140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757c06-f0b2-4374-9ec5-20a8c3613681 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:20.134603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 668/1291] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Aug 25 02:51:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:20.140793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:20.249901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb03461-dbea-451d-9bf9-cd50d4487bfd None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:20.285164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:20.285814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:20.286083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:20.286360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] PATH_INFO: `/v3/services` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:20.287017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:20.287326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:20.289918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e50ce7-9d81-4db4-8793-2978ee2e9fc7 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:20.304182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 624/1292] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Aug 25 02:51:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 02:51:22.205944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f05c3-96c7-4ae2-821a-9c12d2fbd434 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:22.206456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f05c3-96c7-4ae2-821a-9c12d2fbd434 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:22.206456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f05c3-96c7-4ae2-821a-9c12d2fbd434 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:22.207748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 669/1293] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:22.213532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6295a-ba66-475b-ae16-50987036118f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:22.213989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6295a-ba66-475b-ae16-50987036118f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:22.214087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6295a-ba66-475b-ae16-50987036118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:22.232405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-76b6295a-ba66-475b-ae16-50987036118f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:22.233597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-76b6295a-ba66-475b-ae16-50987036118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:22.262022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:22.298409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 625/1294] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:22.567005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f05c3-96c7-4ae2-821a-9c12d2fbd434 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:22.598374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:22.667620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f05c3-96c7-4ae2-821a-9c12d2fbd434 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:22.702735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:22.703342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:22.703636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:22.703890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:22.704567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:22.705910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678cb4-2e27-4966-82c8-a72b5b9e5daf None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:22.847296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 670/1295] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Aug 25 02:51:22 2022] POST /identity/v3/services => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:51:24.793373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f6ef-d309-4383-837c-06c802dec76b None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:24.793902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f6ef-d309-4383-837c-06c802dec76b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:24.794000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f6ef-d309-4383-837c-06c802dec76b None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:24.795278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 626/1296] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:24.800483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:24.800889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:24.801030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:24.819166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:24.820427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:24.892698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 671/1297] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:25.169444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7f6ef-d309-4383-837c-06c802dec76b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:25.273921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7f6ef-d309-4383-837c-06c802dec76b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:25.308172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:25.308792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:25.309078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:25.309360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:25.309942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:25.311311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:25.320516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:25.323777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:25.324897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 627/1298] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:51:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 25 02:51:25.327653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:25.346407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:25.426013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2222ef-256f-4b54-9ab2-ba675d66ffd3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:25.460886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:25.461492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:25.461774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:25.462054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:25.462685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:25.462992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:25.464458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:25.469456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 672/1299] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Aug 25 02:51:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:25.475855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:25.508505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:25.575938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a1e68-108b-4d49-b697-d7d7ed1dee32 None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:25.610068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:25.610679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:25.610962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:25.611240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:25.611943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] RBAC: Adding query filter params (interface=public, service_id=3bf1076d9dc248d2962956731f5b3ab0, region_id=RegionOne) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:25.612289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:25.613745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb1fb9-c493-4f8a-852f-bc7b963c9c45 None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:25.619581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 628/1300] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Aug 25 02:51:25 2022] GET /identity/v3/endpoints?service_id=3bf1076d9dc248d2962956731f5b3ab0&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:25.625245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:25.711006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:25.728126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14472ed9-bdad-4f59-b704-f6e0ddb81d38 None admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:25.761191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:25.761838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:25.762111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:25.762402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:25.762995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:25.764367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc98be3-162e-416d-af2c-07d18c1a6ad3 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:25.770918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 673/1301] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Aug 25 02:51:25 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 02:51:27.668423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3808-931d-4c81-b83f-2e1a036cf17f None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:27.668976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3808-931d-4c81-b83f-2e1a036cf17f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:27.668976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3808-931d-4c81-b83f-2e1a036cf17f None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:27.670366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 629/1302] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:51:27.675541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:27.675921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:27.676086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:27.694308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:51:27.695562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:51:27.760849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 674/1303] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:51:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 25 02:51:28.036599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c3808-931d-4c81-b83f-2e1a036cf17f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:28.130176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:28.137622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c3808-931d-4c81-b83f-2e1a036cf17f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:28.170647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:28.171282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:28.171565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:28.171841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:28.172463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:28.173874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 25 02:51:28.183168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 25 02:51:28.184352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.get_or_create( Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application with Lock( Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self._enter() Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application return self.creator() Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application created_value = creator( Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: ERROR keystone.server.flask.application Aug 25 02:51:28.185323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 630/1304] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Aug 25 02:51:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 25 02:51:28.190248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:28.265154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:28.287102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d52f27-4f75-4d50-be07-589d502de0f1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:28.320507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:28.321112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:28.321392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:28.321694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] PATH_INFO: `/v3/services` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:28.322510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 02:51:28.322968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:28.324769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac3478-5b19-469d-85d0-3f38e4d49586 None admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:28.329825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 675/1305] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Aug 25 02:51:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 02:51:28.336173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:51:28.435412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2ee6-d0ad-492a-9c1a-e9341b4d508c None admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:51:28.468345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:51:28.468943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:51:28.469217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:51:28.469486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:51:28.470132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:51:28.471504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e2b18-764a-48c7-bfc4-fb07e6d694de None admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:51:28.555871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 631/1306] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 02:51:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 02:51:38.326859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=30 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:51:38.550382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:57:55.244910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11b54-f20f-4eef-907b-414387e9c616 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:55.245356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11b54-f20f-4eef-907b-414387e9c616 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:55.245356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11b54-f20f-4eef-907b-414387e9c616 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:55.246684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 676/1307] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:57:55.251895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1045c61-93ea-4814-af29-80ef110f60d6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:55.252394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1045c61-93ea-4814-af29-80ef110f60d6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:55.252527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1045c61-93ea-4814-af29-80ef110f60d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:55.327210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a1045c61-93ea-4814-af29-80ef110f60d6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:57:55.328435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a1045c61-93ea-4814-af29-80ef110f60d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:55.456777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 632/1308] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:57:55.780783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:55.781059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:55.781529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:55.782912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 677/1309] 10.222.0.75 () {54 vars in 965 bytes} [Thu Aug 25 02:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:57:55.789529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb0707-e642-4f3c-a959-b8b7718a36dc None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:55.789857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb0707-e642-4f3c-a959-b8b7718a36dc None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:55.790112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb0707-e642-4f3c-a959-b8b7718a36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:55.869210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-23fb0707-e642-4f3c-a959-b8b7718a36dc None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:57:55.870424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-23fb0707-e642-4f3c-a959-b8b7718a36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:55.891991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:57:55.955393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 633/1310] 10.222.0.75 () {58 vars in 1062 bytes} [Thu Aug 25 02:57:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:57:55.964119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:57:55.965887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:56.094568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156657ed-e4c5-4af9-b52a-f94075ead7eb None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:57:56.181031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:57:56.181564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:56.181840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:56.182110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:56.183746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:56.244779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:57:56.301601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:57:56.303987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d02ed7-4154-4fcb-a59b-4ac30c635855 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:57:56.326675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 678/1311] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:57:58.563322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:58.563882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:58.563882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:58.565906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 634/1312] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:57:58.571440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d618b-9b1e-437d-8b78-6ea525cb5a8f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:58.571892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d618b-9b1e-437d-8b78-6ea525cb5a8f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:58.571998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d618b-9b1e-437d-8b78-6ea525cb5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:58.665345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d15d618b-9b1e-437d-8b78-6ea525cb5a8f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:57:58.666715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d15d618b-9b1e-437d-8b78-6ea525cb5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:58.776009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 679/1313] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:57:59.062330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:57:59.063876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:59.166987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312897f-5426-45c9-a21e-73ee5e66a8fa None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:57:59.238260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:57:59.240831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:57:59.241546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:57:59.241717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:57:59.242013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:57:59.243664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:57:59.351255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:57:59.352748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dff282-ae7b-4933-b1a2-bf996fb98855 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:57:59.371344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 635/1314] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:01.819201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:01.819741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:01.819741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:01.821112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 680/1315] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:01.826299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3ccb-2107-4c4b-a31c-77f7cec47d91 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:01.826673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3ccb-2107-4c4b-a31c-77f7cec47d91 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:01.826854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c3ccb-2107-4c4b-a31c-77f7cec47d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:01.892048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9c6c3ccb-2107-4c4b-a31c-77f7cec47d91 None None] MFA Rules not processed for user `ad5c7bc2451948c38fb9372c8a4cef9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:01.893897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9c6c3ccb-2107-4c4b-a31c-77f7cec47d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:01.944697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:01.988227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 636/1316] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:02.275039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:02.276850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:02.286487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:02.379167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe013f2-815a-4b5f-8d30-515e992a52c4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:02.452951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:02.453556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:02.453834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:02.454107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:02.456354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:02.517980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:02.565213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:02.566554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23598e0-cc9a-4b4b-a7c0-f4b3f5b80274 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:02.585071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 681/1317] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:05.259708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:05.260270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:05.260270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:05.261539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 637/1318] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:05.266819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7171ea-75b3-408a-95ea-83273f928da1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:05.267208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7171ea-75b3-408a-95ea-83273f928da1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:05.268064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7171ea-75b3-408a-95ea-83273f928da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:05.333393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8d7171ea-75b3-408a-95ea-83273f928da1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:05.334603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8d7171ea-75b3-408a-95ea-83273f928da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:05.429732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 682/1319] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:05.720488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:05.722106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:05.825180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2e448-ce88-414c-91e8-feb50e6fc8fa None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:05.898945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:05.899550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:05.899830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:05.900103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:05.901983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:05.909884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:06.011772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:06.013163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2278-584c-4cc9-baaf-19d019fea223 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:06.031301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 638/1320] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:08.978670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a401-df5f-4360-945d-a5baa6e2654c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:08.979197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a401-df5f-4360-945d-a5baa6e2654c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:08.979305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a401-df5f-4360-945d-a5baa6e2654c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:08.980623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 683/1321] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:08.985851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf1e1-bb7c-4a70-9e52-00ada0d31ce3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:08.986303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf1e1-bb7c-4a70-9e52-00ada0d31ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:08.986488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acaf1e1-bb7c-4a70-9e52-00ada0d31ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:09.061682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1acaf1e1-bb7c-4a70-9e52-00ada0d31ce3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:09.062887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1acaf1e1-bb7c-4a70-9e52-00ada0d31ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:09.138027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:09.179286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 639/1322] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:58:09.483481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:09.483772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:09.484048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:09.485431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 684/1323] 10.222.0.75 () {54 vars in 965 bytes} [Thu Aug 25 02:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:09.491963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139959-48e7-41b3-8cfe-de3a8b832bc0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:09.501187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139959-48e7-41b3-8cfe-de3a8b832bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:09.501187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139959-48e7-41b3-8cfe-de3a8b832bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:09.570049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-13139959-48e7-41b3-8cfe-de3a8b832bc0 None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:09.571253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-13139959-48e7-41b3-8cfe-de3a8b832bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:09.649097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 640/1324] 10.222.0.75 () {58 vars in 1062 bytes} [Thu Aug 25 02:58:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:09.658751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:09.660293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:09.679340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:09.765216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e38b2-0d74-4f26-974d-d7c49e30582c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:09.841642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:09.842251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:09.842528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:09.842798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:09.844420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:09.921671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:09.961183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:09.962612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1dc6b-13a3-45bb-9728-e703a1dd0253 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:09.981803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 685/1325] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:12.195918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:12.196503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:12.196503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:12.197804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 641/1326] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:12.203762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69baf-81e3-4878-a020-1dc5c2baea73 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:12.204333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69baf-81e3-4878-a020-1dc5c2baea73 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:12.204333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69baf-81e3-4878-a020-1dc5c2baea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:12.273802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-85b69baf-81e3-4878-a020-1dc5c2baea73 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:12.275021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-85b69baf-81e3-4878-a020-1dc5c2baea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:12.370978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 686/1327] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:12.659620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:12.661273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:12.723814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:12.772832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1711-0ba2-4fa4-8eaa-0511382e555a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:12.847789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:12.848491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:12.848773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:12.849042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:12.850673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:12.965142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:12.967512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:12.969043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a46f3-56db-4a02-a0b2-79c52d111841 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:12.989415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 642/1328] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:14.963342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:14.963842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:14.963940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:14.965263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 687/1329] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:14.970412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3be5d-ab00-474e-a04c-7dd4fc32438b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:14.970818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3be5d-ab00-474e-a04c-7dd4fc32438b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:14.971062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3be5d-ab00-474e-a04c-7dd4fc32438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:15.043863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0db3be5d-ab00-474e-a04c-7dd4fc32438b None None] MFA Rules not processed for user `ad5c7bc2451948c38fb9372c8a4cef9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:15.045315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0db3be5d-ab00-474e-a04c-7dd4fc32438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:15.169985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 643/1330] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:15.454931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:15.456566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:15.489101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:15.562845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db606ef2-469e-40b0-b8d6-e7171bb6e098 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:15.638155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:15.638766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:15.639048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:15.639325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:15.640999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:15.725396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:15.761030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:15.762459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2a64-afed-4483-8776-e636433cc49f service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:15.782156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 688/1331] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:18.327502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:18.328010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:18.328010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:18.329956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 644/1332] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:18.335418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c832c-0cab-4cf5-9602-5eec181cef1a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:18.335859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c832c-0cab-4cf5-9602-5eec181cef1a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:18.335977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c832c-0cab-4cf5-9602-5eec181cef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:18.401126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-095c832c-0cab-4cf5-9602-5eec181cef1a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:18.402359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-095c832c-0cab-4cf5-9602-5eec181cef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:18.498722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 689/1333] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:18.781270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:18.783094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:18.852534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:18.886895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c5676-76c3-42f4-a5ef-6f136831f899 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:18.962166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:18.962773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:18.963051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:18.963321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:18.964964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:19.079141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:19.080701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade0fc-4718-4483-a774-ca525a52cef8 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:19.090424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:19.098957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 645/1334] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:21.096575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:21.097094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:21.097192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:21.098506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 690/1335] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:21.103762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51c63e-44ce-442e-bf3f-dc834c9fc0e4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:21.104201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51c63e-44ce-442e-bf3f-dc834c9fc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:21.104454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51c63e-44ce-442e-bf3f-dc834c9fc0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:21.172331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4d51c63e-44ce-442e-bf3f-dc834c9fc0e4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:21.175444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4d51c63e-44ce-442e-bf3f-dc834c9fc0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:21.272428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 646/1336] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:21.566155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:21.567687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:21.608467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:21.670170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24209bc-3531-44a2-bc9f-8f465690dc0d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:21.740944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:21.741539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:21.741815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:21.742083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:21.743654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:21.826704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:21.852838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:21.854259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78471d53-4a48-431e-99f0-cee07bdbd88b service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:21.872000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 691/1337] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:24.342000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:24.342563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:24.342563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:24.343790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 647/1338] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:24.349503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6f78-19fb-4051-82a5-d8db53641d22 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:24.349923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6f78-19fb-4051-82a5-d8db53641d22 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:24.350089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6f78-19fb-4051-82a5-d8db53641d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:24.416319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-51dc6f78-19fb-4051-82a5-d8db53641d22 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:24.417521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-51dc6f78-19fb-4051-82a5-d8db53641d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:24.514806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 692/1339] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:24.801950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:24.803607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:24.877817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:24.904963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd463af-c1df-493b-8a86-47bb10b321ae None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:24.980346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:24.980955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:24.981235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:24.981504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:24.983110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:25.091285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:25.092762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a5080-31c0-45f0-a1d5-d95a728eaa7b service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:25.111909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 648/1340] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:29.274153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:29.274678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:29.274678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:29.276256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 693/1341] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:29.281740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb75e9-87af-4135-a518-2ad23b3014db None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:29.282185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb75e9-87af-4135-a518-2ad23b3014db None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:29.282358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb75e9-87af-4135-a518-2ad23b3014db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:29.288396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:29.354965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-45bb75e9-87af-4135-a518-2ad23b3014db None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:29.356243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-45bb75e9-87af-4135-a518-2ad23b3014db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:29.459322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 649/1342] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:29.748165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:29.749761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:29.799036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:29.855041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a1b6-6f4d-4d9a-9d58-0cfa7d71e459 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:29.930908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:29.932019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:29.932019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:29.932209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:29.933777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:30.028876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:30.045172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:30.046599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87408e-9066-40d8-8397-aaf009917acf service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:30.065306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 694/1343] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:32.961212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:32.961743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:32.961743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:32.963256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 650/1344] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:32.968351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9beae-7514-4a56-a322-df7f60bb693c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:32.968775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9beae-7514-4a56-a322-df7f60bb693c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:32.968911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9beae-7514-4a56-a322-df7f60bb693c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:33.037539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-18d9beae-7514-4a56-a322-df7f60bb693c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:33.038638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-18d9beae-7514-4a56-a322-df7f60bb693c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:33.139252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 695/1345] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:33.433395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:33.435024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:33.530947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:33.549317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98409224-4f0a-4a7b-adf9-d591eaf65ae9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:33.629337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:33.629868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:33.630151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:33.630426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:33.632193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:33.753035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:33.754516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a389e-8cad-42aa-b876-47e829995dff service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:33.776555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 651/1346] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:38.282728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:38.283224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:38.283319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:38.284650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 696/1347] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:38.290186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57efed-96ef-4787-99e7-fb913920979f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:38.290661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57efed-96ef-4787-99e7-fb913920979f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:38.290759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57efed-96ef-4787-99e7-fb913920979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:38.307504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:38.364338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1d57efed-96ef-4787-99e7-fb913920979f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:38.365570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1d57efed-96ef-4787-99e7-fb913920979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:38.470391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 652/1348] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:38.756986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:38.758500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:38.817848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:38.864638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442f7e5-204b-4d6a-8c75-43f90fff1f2b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:38.941249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:38.941861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:38.942144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:38.942419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:38.944053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:39.050327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:39.053432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:39.054865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a381c-c9f9-440a-96bc-f789964228cb service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:39.078885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 697/1349] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:44.005452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7c0c-af47-4f8a-a238-ae63c3c2787f None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:44.005986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7c0c-af47-4f8a-a238-ae63c3c2787f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:44.006107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b7c0c-af47-4f8a-a238-ae63c3c2787f None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:44.007403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 653/1350] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 02:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:44.012576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c7f-eadd-46af-a3a5-75198da6cbbe None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:44.013039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c7f-eadd-46af-a3a5-75198da6cbbe None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:44.013375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c7f-eadd-46af-a3a5-75198da6cbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:44.105483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-cf1d2c7f-eadd-46af-a3a5-75198da6cbbe None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:44.107045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cf1d2c7f-eadd-46af-a3a5-75198da6cbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:44.286056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 698/1351] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 02:58:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:48.467852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=59 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:48.726947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269dfb-18f2-4241-8bd7-3f002472c602 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:48.727137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269dfb-18f2-4241-8bd7-3f002472c602 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:48.727459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269dfb-18f2-4241-8bd7-3f002472c602 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:48.729098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 654/1352] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 02:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:48.734112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c9f2-a28a-47e9-bb14-9f0f493547d9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:48.734602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c9f2-a28a-47e9-bb14-9f0f493547d9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:48.734868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c9f2-a28a-47e9-bb14-9f0f493547d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:48.791895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:48.815190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fa78c9f2-a28a-47e9-bb14-9f0f493547d9 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:48.816499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fa78c9f2-a28a-47e9-bb14-9f0f493547d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:48.943245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 699/1353] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 02:58:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:50.218327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d88b-0200-41c0-9ff2-638ebc1c3188 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:50.219055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d88b-0200-41c0-9ff2-638ebc1c3188 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:50.219055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d88b-0200-41c0-9ff2-638ebc1c3188 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:50.220280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 655/1354] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 02:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:50.225565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c614-9215-4d72-97dd-6b6e2ef369c9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:50.226036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c614-9215-4d72-97dd-6b6e2ef369c9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:50.226134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c614-9215-4d72-97dd-6b6e2ef369c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:50.302307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1265c614-9215-4d72-97dd-6b6e2ef369c9 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:50.303546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1265c614-9215-4d72-97dd-6b6e2ef369c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:50.347615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:50.417750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 700/1355] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 02:58:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:53.723601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437877ef-94ab-4f99-8e79-37a7f35f578c None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:53.724180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437877ef-94ab-4f99-8e79-37a7f35f578c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:53.724180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437877ef-94ab-4f99-8e79-37a7f35f578c None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:53.725504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 656/1356] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:53.730867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8d0a-c8c9-4647-a969-c3149a92a0ff None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:53.731273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8d0a-c8c9-4647-a969-c3149a92a0ff None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:53.731414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8d0a-c8c9-4647-a969-c3149a92a0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:53.816485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-34bb8d0a-c8c9-4647-a969-c3149a92a0ff None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:53.817711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-34bb8d0a-c8c9-4647-a969-c3149a92a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:53.934422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 701/1357] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:58:54.252741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:54.253244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:54.253685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:54.255515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 657/1358] 10.222.0.75 () {54 vars in 962 bytes} [Thu Aug 25 02:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:54.265702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167794a6-17f4-4ff3-bc5b-90f976df3ddc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:54.266345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167794a6-17f4-4ff3-bc5b-90f976df3ddc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:54.276487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167794a6-17f4-4ff3-bc5b-90f976df3ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:54.288772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:54.373329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-167794a6-17f4-4ff3-bc5b-90f976df3ddc None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:54.375231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-167794a6-17f4-4ff3-bc5b-90f976df3ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:54.503889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 702/1359] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Aug 25 02:58:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:58:54.511728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:54.513609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:54.696267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35072c1-241c-4f4b-824a-6c825bfea2de None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:54.774851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:54.815951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:54.816718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:54.817172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:54.817526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:54.820284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:54.914799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:54.916714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07433f52-bdf3-43af-90b8-9c0cc154ba9a service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:54.937691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 658/1360] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:58:57.129524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:57.130155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:57.130155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:57.131704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 703/1361] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:58:57.136992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899c6-059c-4454-b826-ee5499c5ed88 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:57.137407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899c6-059c-4454-b826-ee5499c5ed88 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:57.137559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870899c6-059c-4454-b826-ee5499c5ed88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:57.207681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-870899c6-059c-4454-b826-ee5499c5ed88 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:58:57.208911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-870899c6-059c-4454-b826-ee5499c5ed88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:57.248291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:57.320336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 659/1362] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:58:57.627650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:58:57.632855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:57.707637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:57.795211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf563a40-2682-4d91-af45-2a38af04d167 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:58:57.919971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:58:57.920538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:58:57.920815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:58:57.921086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:58:57.925404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:58:57.952396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:58:58.028634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:58:58.030290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9192e0b-ee93-4de8-ac58-e5309513caeb service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:58:58.051393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 704/1363] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:00.444482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797739-6955-4a69-bb73-2ea74fe883eb None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:00.445016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797739-6955-4a69-bb73-2ea74fe883eb None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:00.445016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797739-6955-4a69-bb73-2ea74fe883eb None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:00.446387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 660/1364] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:00.451873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0f4f-4906-411d-b088-a99c081a9636 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:00.452695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0f4f-4906-411d-b088-a99c081a9636 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:00.452695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0f4f-4906-411d-b088-a99c081a9636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:00.522364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-42aa0f4f-4906-411d-b088-a99c081a9636 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:00.523575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-42aa0f4f-4906-411d-b088-a99c081a9636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:00.631646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:00.636197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 705/1365] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:00.931907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:00.932563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:00.933042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:00.935235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 661/1366] 10.222.0.75 () {54 vars in 962 bytes} [Thu Aug 25 02:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:00.941161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cbb-be7a-41d2-b8c5-2ffca61376a4 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:00.941551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cbb-be7a-41d2-b8c5-2ffca61376a4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:00.941939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cbb-be7a-41d2-b8c5-2ffca61376a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:01.027270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f91f2cbb-be7a-41d2-b8c5-2ffca61376a4 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:01.028798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f91f2cbb-be7a-41d2-b8c5-2ffca61376a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:01.159471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 706/1367] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Aug 25 02:59:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:01.167531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:01.169138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:01.326675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c7149-0aba-4892-acca-8e498ea2d487 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:01.349184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:01.446382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:01.447033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:01.447406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:01.447771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:01.449478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:01.551181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:01.552959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b93283-a816-4bd9-9632-b54be3719849 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:01.571991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 662/1368] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:02.084528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6ba9e-a451-4383-8aad-b84a8d418a22 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:02.085064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6ba9e-a451-4383-8aad-b84a8d418a22 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:02.085064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6ba9e-a451-4383-8aad-b84a8d418a22 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:02.086507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 707/1369] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:02.091863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b069f49-4d92-4330-a01e-073221af543f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:02.092192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b069f49-4d92-4330-a01e-073221af543f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:02.092464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b069f49-4d92-4330-a01e-073221af543f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:02.126766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:02.166418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9b069f49-4d92-4330-a01e-073221af543f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:02.167949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9b069f49-4d92-4330-a01e-073221af543f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:02.274097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 663/1370] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:02.872776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:02.872776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:02.872776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:02.876022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 708/1371] 10.222.0.75 () {54 vars in 964 bytes} [Thu Aug 25 02:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:02.880217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5068e3-ae41-414a-982d-f97329929258 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:02.881038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5068e3-ae41-414a-982d-f97329929258 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:02.881038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5068e3-ae41-414a-982d-f97329929258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:02.961627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-eb5068e3-ae41-414a-982d-f97329929258 None None] MFA Rules not processed for user `57f08a89a0dd4befb153c2b38167b5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:02.963824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eb5068e3-ae41-414a-982d-f97329929258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:03.016407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:03.051028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 664/1372] 10.222.0.75 () {58 vars in 1061 bytes} [Thu Aug 25 02:59:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:03.058801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:03.060577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:03.114874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:03.172058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dc098-c6d1-4211-944f-13ac879222f9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:03.253693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:03.254306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:03.254586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:03.254861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:03.256538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:03.363015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:03.379100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:03.381098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13763f-6350-467f-a51a-602ff85c5ba3 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:03.407829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 709/1373] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:03.731216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:03.731622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:03.731900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:03.733290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 665/1374] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:03.738679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb2-fe3e-49a4-b6af-99a50f97b18f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:03.738967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb2-fe3e-49a4-b6af-99a50f97b18f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:03.739241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb2-fe3e-49a4-b6af-99a50f97b18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:03.806289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d0a70fb2-fe3e-49a4-b6af-99a50f97b18f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:03.807487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d0a70fb2-fe3e-49a4-b6af-99a50f97b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:03.904628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 710/1375] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:04.202748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:04.204449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:04.354323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c801d1a-28ee-4ceb-9cca-af7fa44c338c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:04.406500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:04.469622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:04.470178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:04.470458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:04.470729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:04.472443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:04.560462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:04.562634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e96b9f-b8cc-4dfc-a9de-08dcdb535308 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:04.582207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 666/1376] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:06.766364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:06.766869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:06.766869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:06.768295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 711/1377] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:06.773631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f857c3d-94b0-4378-8370-4b4d16c4e0e3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:06.774275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f857c3d-94b0-4378-8370-4b4d16c4e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:06.774562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f857c3d-94b0-4378-8370-4b4d16c4e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:06.842149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2f857c3d-94b0-4378-8370-4b4d16c4e0e3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:06.843638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2f857c3d-94b0-4378-8370-4b4d16c4e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:06.850915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:06.949455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 667/1378] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:07.272496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:07.274244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:07.342313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:07.441790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26accf-3fef-4f7f-87fb-628f1bbe32e5 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:07.572419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:07.573191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:07.573575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:07.573896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:07.575710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:07.602109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:07.683002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:07.684748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53525b38-b1c8-4b4a-bbf3-d011f99cf3d6 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:07.705047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 712/1379] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:09.853526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:09.854055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:09.854055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:09.855464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 668/1380] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:09.860844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24868-b342-474e-a1e3-ef7554cccd9f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:09.861280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24868-b342-474e-a1e3-ef7554cccd9f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:09.861368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c24868-b342-474e-a1e3-ef7554cccd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:09.939452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-92c24868-b342-474e-a1e3-ef7554cccd9f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:09.940879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-92c24868-b342-474e-a1e3-ef7554cccd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:10.061241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:10.068944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 713/1381] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:10.369508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:10.370994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:10.521923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:10.528906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fe61f-01bc-4d13-a365-3fa01c39ecd6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:10.652428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:10.653032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:10.653311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:10.653581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:10.655680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:10.751096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:10.752838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a3f9a-34e4-4601-a409-3ae96596f018 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:10.765042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:10.776327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 669/1382] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:13.260533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:13.261223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:13.261223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:13.262358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 714/1383] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:13.269104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff8d4-d2b0-4ec9-87f7-b973d486d485 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:13.269567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff8d4-d2b0-4ec9-87f7-b973d486d485 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:13.269835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff8d4-d2b0-4ec9-87f7-b973d486d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:13.354417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a69ff8d4-d2b0-4ec9-87f7-b973d486d485 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:13.355970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a69ff8d4-d2b0-4ec9-87f7-b973d486d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:13.464421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 670/1384] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:13.760245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:13.761746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:13.911100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977e7ee-7237-4c34-ac65-19e1dee9c7a8 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:13.981369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:14.038668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:14.039422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:14.039792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:14.040138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:14.042250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:14.143080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:14.144872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f5905-0f7c-4ca9-93c2-d0009a94a394 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:14.165299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 715/1385] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:16.107957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:16.109306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:16.109306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:16.110558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 671/1386] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:16.115954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1785b-e1ff-4583-894c-92c4394a660b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:16.116472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1785b-e1ff-4583-894c-92c4394a660b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:16.116601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1785b-e1ff-4583-894c-92c4394a660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:16.184026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fca1785b-e1ff-4583-894c-92c4394a660b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:16.185203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fca1785b-e1ff-4583-894c-92c4394a660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:16.220280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:16.288572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 716/1387] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:16.582625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:16.584615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:16.666342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:16.756558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1c4db-6eb1-4077-b5af-14633c7fef7c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:16.874486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:16.875135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:16.875354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:16.875638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:16.877496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:16.919057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:16.976226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:16.977911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd33d20-8ff2-43f1-bf7b-5ff2a6c54873 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:16.996817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 672/1388] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:19.069946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665a42-dce1-42a7-bf2b-de430cd6722c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.070505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665a42-dce1-42a7-bf2b-de430cd6722c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.070505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665a42-dce1-42a7-bf2b-de430cd6722c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.071973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 717/1389] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:19.077438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f790-6de2-4a31-a812-72120d5badfb None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.078473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f790-6de2-4a31-a812-72120d5badfb None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.078657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f790-6de2-4a31-a812-72120d5badfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.104014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b56077-e356-4967-aa3b-1c5e25d87151 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.104332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b56077-e356-4967-aa3b-1c5e25d87151 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.104734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b56077-e356-4967-aa3b-1c5e25d87151 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.106077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 718/1390] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 02:59:19.111543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7a0ff-2c3b-42dd-85df-26598fdd33c7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.111861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7a0ff-2c3b-42dd-85df-26598fdd33c7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.112158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7a0ff-2c3b-42dd-85df-26598fdd33c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.146131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cda2f790-6de2-4a31-a812-72120d5badfb None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:19.147334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cda2f790-6de2-4a31-a812-72120d5badfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.183215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-46b7a0ff-2c3b-42dd-85df-26598fdd33c7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:19.184360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-46b7a0ff-2c3b-42dd-85df-26598fdd33c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.247579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 673/1391] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:19.285529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 719/1392] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:59:19.553651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.553943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.554217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.555596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 674/1393] 10.222.0.75 () {54 vars in 967 bytes} [Thu Aug 25 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:19.560606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.560895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.561172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.578126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:19.585667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:19.587173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.602120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:19.641864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] MFA Rules not processed for user `ca96b68db5ab4c42ae4d34f6ce0a21d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:19.643080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.721032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 720/1394] 10.222.0.75 () {58 vars in 1064 bytes} [Thu Aug 25 02:59:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:19.728796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:19.737103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d434a3c-81b4-4dca-94d3-8f0477cf2ece None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:19.737809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042fe3a-790e-490a-afcb-81568e865cf9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:19.813932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:19.814451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.814734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.815009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.816646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.826256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:19.847646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:19.855003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:19.855603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:19.855875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:19.856140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:19.857747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:19.931008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:19.932483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b2732-8995-4401-83f1-2e8fb5f4f1c8 service glance] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:19.954580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 721/1395] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:19.958141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:19.959763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb9967-9f9d-492b-8b77-6c3f37e74733 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:19.979409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 675/1396] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 02:59:20.900294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0b724-c882-49c7-884a-a6e6c895e9c9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:20.900771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0b724-c882-49c7-884a-a6e6c895e9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:20.900973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0b724-c882-49c7-884a-a6e6c895e9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:20.973943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f8f0b724-c882-49c7-884a-a6e6c895e9c9 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:20.975177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f8f0b724-c882-49c7-884a-a6e6c895e9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:21.056373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:21.062706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 722/1397] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 02:59:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:21.110673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:21.111136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:21.111228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:21.112600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 676/1398] 10.222.0.75 () {54 vars in 943 bytes} [Thu Aug 25 02:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:21.119021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e198b-96bd-4187-be7d-16c4ef47b9e8 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:21.119313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e198b-96bd-4187-be7d-16c4ef47b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:21.119618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e198b-96bd-4187-be7d-16c4ef47b9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:21.197506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a73e198b-96bd-4187-be7d-16c4ef47b9e8 None None] MFA Rules not processed for user `a8798813bd1e461d81faf73696f62a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:21.198631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a73e198b-96bd-4187-be7d-16c4ef47b9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:21.307325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 723/1399] 10.222.0.75 () {58 vars in 1040 bytes} [Thu Aug 25 02:59:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:21.316975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:21.318534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:21.416625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:21.457117 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7794bf4-c66f-4f7b-b329-31ce15718bdf None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:21.566660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:21.567264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:21.567537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:21.567803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:21.569498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:21.659786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:21.661613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38e947-f614-48e4-80c8-8ce0715c2058 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:21.665356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:21.670378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 677/1400] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 02:59:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 02:59:22.136333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:22.137022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:22.137285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:22.138997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 724/1401] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:22.149344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd7fb-1c0a-48f4-bf41-d348d473253c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:22.151523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd7fb-1c0a-48f4-bf41-d348d473253c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:22.151523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd7fb-1c0a-48f4-bf41-d348d473253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:22.227384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-70dfd7fb-1c0a-48f4-bf41-d348d473253c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:22.228647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-70dfd7fb-1c0a-48f4-bf41-d348d473253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:22.335220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 678/1402] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:22.637238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:22.638830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:22.705066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:22.807428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dfa63-c3fc-43ad-95b5-89d26eab1717 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:22.937582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:22.938445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:22.938723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:22.939000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:22.940682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:22.961094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:23.042494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:23.044100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999705f8-b219-4efb-8d90-0d6a300d669b service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:23.067358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 725/1403] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:25.223538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:25.224110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:25.224110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:25.225536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 679/1404] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:25.230875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84248-4039-4a51-8b28-9d5415e57360 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:25.231303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84248-4039-4a51-8b28-9d5415e57360 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:25.231405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f84248-4039-4a51-8b28-9d5415e57360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:25.319471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b6f84248-4039-4a51-8b28-9d5415e57360 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:25.320772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b6f84248-4039-4a51-8b28-9d5415e57360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:25.419844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:25.429343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 726/1405] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:25.730302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:25.732111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:25.822627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:25.900375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76505318-6b22-40b1-8065-abf9790fa6c6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:26.045848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:26.047435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:26.047915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:26.048211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:26.051406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:26.103876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:26.159609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:26.161154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6b39-43b5-4f33-a2c0-5fa716bfe928 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:26.181303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 680/1406] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:28.285044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:28.285560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:28.285560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:28.287026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 727/1407] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 02:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:28.292587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70dce-ebd1-42b0-ab60-7a75f4e779d6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:28.293029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70dce-ebd1-42b0-ab60-7a75f4e779d6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:28.293204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70dce-ebd1-42b0-ab60-7a75f4e779d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:28.365103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b7c70dce-ebd1-42b0-ab60-7a75f4e779d6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:28.366324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b7c70dce-ebd1-42b0-ab60-7a75f4e779d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:28.477726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 681/1408] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 02:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:28.774912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:28.776512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:28.948975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257680c-8d4e-4e1c-bf51-a3a8d07b88df None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:29.025712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:29.083691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:29.084344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:29.084629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:29.084903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:29.086503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:29.187880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:29.188557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7f2fd-3150-4b4e-ab6c-e0ac21b4de85 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:29.207825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 728/1409] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:38.475278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:39.205660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=65 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:45.368706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8f49-7a81-4d36-9aba-f212f0195192 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.369252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8f49-7a81-4d36-9aba-f212f0195192 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.369252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c8f49-7a81-4d36-9aba-f212f0195192 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.370737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 682/1410] 10.222.0.75 () {54 vars in 918 bytes} [Thu Aug 25 02:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:45.375764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a481c-2a1f-498d-a051-7b48622322dc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.376205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a481c-2a1f-498d-a051-7b48622322dc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.376337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a481c-2a1f-498d-a051-7b48622322dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.450718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c00a481c-2a1f-498d-a051-7b48622322dc None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:45.452609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c00a481c-2a1f-498d-a051-7b48622322dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:45.558559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 729/1411] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Aug 25 02:59:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:45.572773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bbd0-2060-4d9f-9d73-4e638bb99c54 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.573058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bbd0-2060-4d9f-9d73-4e638bb99c54 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.573730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bbd0-2060-4d9f-9d73-4e638bb99c54 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.574699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 683/1412] 10.222.0.75 () {54 vars in 918 bytes} [Thu Aug 25 02:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:45.580491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84c1c-fed5-4d4c-9fdc-cdec52f21412 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:45.580779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84c1c-fed5-4d4c-9fdc-cdec52f21412 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:45.581055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84c1c-fed5-4d4c-9fdc-cdec52f21412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:45.654316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:45.659172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4cb84c1c-fed5-4d4c-9fdc-cdec52f21412 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:45.660419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4cb84c1c-fed5-4d4c-9fdc-cdec52f21412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:45.774533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 730/1413] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Aug 25 02:59:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:48.183056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:48.183556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:48.183556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:48.184922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 684/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 02:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:48.190332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39bb2-5612-4422-a96e-b650200dd1df None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:48.190769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39bb2-5612-4422-a96e-b650200dd1df None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:48.190880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e39bb2-5612-4422-a96e-b650200dd1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:48.257343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-22e39bb2-5612-4422-a96e-b650200dd1df None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:48.258543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-22e39bb2-5612-4422-a96e-b650200dd1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:48.312618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:48.355200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 731/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 02:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:48.645090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:48.646640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:48.792945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c3103-3fe7-4dd2-9408-01fc373533e9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:48.866352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:48.906747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:48.907271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:48.907553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:48.907819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:48.909565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:48.998991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:49.000624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a5985-472a-4d1e-b83a-1a5319a69d16 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:49.018449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 685/1416] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:49.907891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.908523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.908523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.909868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 732/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:49.914824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafdd2-ce22-48fd-a229-4dfbfc36181d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:49.915232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafdd2-ce22-48fd-a229-4dfbfc36181d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:49.915407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafdd2-ce22-48fd-a229-4dfbfc36181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:49.992235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-addafdd2-ce22-48fd-a229-4dfbfc36181d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:49.993435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-addafdd2-ce22-48fd-a229-4dfbfc36181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:50.020843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.097221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 686/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:50.262006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:50.263424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:50.416437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519322-ec1f-4117-a309-8a79a04345da None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:50.459234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:50.539357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:50.539895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:50.540197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:50.540482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:50.542078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:50.640767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:50.642360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74533d43-9669-4a71-b884-c1a0b73ba2ed service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:50.662511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 733/1419] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:51.567910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.568541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.568541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.569847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 687/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:51.574755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647081-633c-45bd-bbea-a0e8c11647ca None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:51.582643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647081-633c-45bd-bbea-a0e8c11647ca None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:51.582643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647081-633c-45bd-bbea-a0e8c11647ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:51.632749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:51.656836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-80647081-633c-45bd-bbea-a0e8c11647ca None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:51.658047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-80647081-633c-45bd-bbea-a0e8c11647ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:51.755153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 734/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:51.908243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:51.909862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:52.057725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9f1ab-0587-46e0-b27c-9a949cfa6291 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:52.070168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:52.171660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:52.172240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:52.172521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:52.172792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:52.174384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:52.264735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:52.266353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396416fa-cca9-4f13-884a-5369af389fd0 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:52.284813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 688/1422] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:53.155021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.155497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.155604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.156954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 735/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:53.161916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174001e-58e0-4e2a-aa9a-3d1ee9b65759 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.162319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174001e-58e0-4e2a-aa9a-3d1ee9b65759 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.162466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174001e-58e0-4e2a-aa9a-3d1ee9b65759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.182324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:53.239573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a174001e-58e0-4e2a-aa9a-3d1ee9b65759 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:53.240808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a174001e-58e0-4e2a-aa9a-3d1ee9b65759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:53.337711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 689/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:59:53.494840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:53.496293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:53.605259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:53.643907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad12ce-55ea-44c9-af55-978c5b00aef3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:53.757628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:53.758146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:53.758419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:53.758688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:53.760287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:53.836423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:53.850250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:53.851835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ae304-1e91-4964-b5e9-a529c9af8cb7 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:53.869653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 736/1425] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:54.745699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.746242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.746242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.747481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 690/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:54.753129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4e57-0eaa-44ad-9f9d-2c7f30761a76 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:54.756181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4e57-0eaa-44ad-9f9d-2c7f30761a76 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:54.756181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4e57-0eaa-44ad-9f9d-2c7f30761a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:54.830878 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-88bc4e57-0eaa-44ad-9f9d-2c7f30761a76 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:54.832085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-88bc4e57-0eaa-44ad-9f9d-2c7f30761a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:54.926610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 737/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:55.077487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:55.078903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:55.161763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:55.224695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acff3e-0c57-4afb-8990-cf8ba6620a38 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:55.336745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:55.337274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:55.337555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:55.337830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:55.339439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:55.388998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:55.430728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:55.432352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0599-18d3-422e-a6cd-fcd2a7e83202 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:55.449825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 691/1428] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:56.305785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:56.306302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:56.306302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:56.307610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 738/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:56.312917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c957e2-0467-4bd7-a2fb-f17afbb7fa18 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:56.313332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c957e2-0467-4bd7-a2fb-f17afbb7fa18 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:56.313460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c957e2-0467-4bd7-a2fb-f17afbb7fa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:56.395102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e0c957e2-0467-4bd7-a2fb-f17afbb7fa18 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:56.396379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e0c957e2-0467-4bd7-a2fb-f17afbb7fa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:56.495188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 692/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:56.648688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:56.654665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:56.700700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:56.801032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7bd2a-2422-4345-aa4a-c7886efbc4da None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:56.912757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:56.913281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:56.913560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:56.913829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:56.915431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:56.922686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:57.006971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:57.008721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689bb6a-3f46-4825-b6bc-388e7a46a401 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:57.027192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 739/1431] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:57.915201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:57.915784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:57.915784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:57.917118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 693/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:57.922165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6ad38-c2f7-4d0f-801a-8bfc34712a61 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:57.922613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6ad38-c2f7-4d0f-801a-8bfc34712a61 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:57.922724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6ad38-c2f7-4d0f-801a-8bfc34712a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:57.999145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-40b6ad38-c2f7-4d0f-801a-8bfc34712a61 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:58.000397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-40b6ad38-c2f7-4d0f-801a-8bfc34712a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:58.067771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:58.095379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 740/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 02:59:58.245923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:58.247338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:58.265084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:58.392818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d102-ddfe-4285-ba89-c0ead10fdfff None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 02:59:58.479796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:58.505926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 02:59:58.506456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:58.506736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:58.507008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:58.508915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:58.601083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 02:59:58.602687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306f097-0b28-4246-a7a6-f3461b0b3cb5 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 02:59:58.621438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 694/1434] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 02:59:59.490919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:59.491464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:59.491464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:59.492860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 741/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 02:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 02:59:59.497832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171743a-4cbf-4a4e-8fbb-9d1840eccc7c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 02:59:59.498353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171743a-4cbf-4a4e-8fbb-9d1840eccc7c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 02:59:59.498642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171743a-4cbf-4a4e-8fbb-9d1840eccc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 02:59:59.574194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3171743a-4cbf-4a4e-8fbb-9d1840eccc7c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 02:59:59.580262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3171743a-4cbf-4a4e-8fbb-9d1840eccc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:59.618260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 02:59:59.673625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 695/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 02:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 02:59:59.823306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 02:59:59.824767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 02:59:59.975323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3f72f-e42e-4583-ab24-b9bed198fe19 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:00.031758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:00.092928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:00.093454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:00.093736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:00.094007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:00.095588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:00.185001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:00.187105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c167b2-48c2-44a7-956f-4d0595cc607b service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:00.205111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 742/1437] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 02:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:01.080637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.081226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.081226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.082540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 696/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:01.087654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e4ae-dc45-4c3c-a127-b86fb30c581f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.088064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e4ae-dc45-4c3c-a127-b86fb30c581f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.088213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e4ae-dc45-4c3c-a127-b86fb30c581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.164429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0e64e4ae-dc45-4c3c-a127-b86fb30c581f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:01.165626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0e64e4ae-dc45-4c3c-a127-b86fb30c581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.167575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.265157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 743/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:01.418678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:01.420280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.564573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d64d-85d0-4bf1-b134-2b29f91ed4cd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:01.591824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:01.677875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:01.678468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:01.678740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:01.679002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:01.680666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:01.769749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:01.771368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2714c8-ed7c-4db7-b824-28224ead8bc9 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:01.795543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 697/1440] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:03.621061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.621589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.621589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.623600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 744/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:03.629215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38e59e-13c8-47ba-80be-dfc03113c7aa None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:03.629646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38e59e-13c8-47ba-80be-dfc03113c7aa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:03.629796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38e59e-13c8-47ba-80be-dfc03113c7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:03.671178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:03.696279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3e38e59e-13c8-47ba-80be-dfc03113c7aa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:03.697526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3e38e59e-13c8-47ba-80be-dfc03113c7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:03.795413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 698/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:04.082552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:04.084084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:04.187630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e3e8-5266-40dd-b876-9289ce695054 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:04.219536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:04.260613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:04.261226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:04.261484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:04.261749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:04.263412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:04.371591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:04.373750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041999b5-323e-44ef-9e15-1d0c11b9b624 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:04.393161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 745/1443] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:06.452253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.452783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.452905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.454159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 699/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:06.459739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0809-d4f9-4713-8895-123e1d3e54b1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:06.460193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0809-d4f9-4713-8895-123e1d3e54b1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:06.460341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0809-d4f9-4713-8895-123e1d3e54b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:06.526326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-47ad0809-d4f9-4713-8895-123e1d3e54b1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:06.527537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-47ad0809-d4f9-4713-8895-123e1d3e54b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:06.540890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:06.625841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 746/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:06.914245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:06.915781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:07.023508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faab34-d829-4bc0-852b-4b3f757882a3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:07.035906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:07.097550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:07.098153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:07.098428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:07.098702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:07.100320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:07.209605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:07.210957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433885a8-2afe-4e6b-bd37-ea916c978694 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:07.353067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 700/1446] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:09.630301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.630840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.630840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.632217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 747/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:09.637621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd78425-612d-428b-afd6-0a8e68f17ba2 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:09.638761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd78425-612d-428b-afd6-0a8e68f17ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:09.638862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd78425-612d-428b-afd6-0a8e68f17ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:09.686284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:09.704054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3bd78425-612d-428b-afd6-0a8e68f17ba2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:09.705334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3bd78425-612d-428b-afd6-0a8e68f17ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:09.801961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 701/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:10.093177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:10.094773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:10.200103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a543669-b12a-4604-8e60-ee8437d7bd27 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:10.240976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:10.276314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:10.276919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:10.277200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:10.277474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:10.279069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:10.387570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:10.389060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b54ab-e369-446f-a5d3-d9b8450412f5 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:10.407921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 748/1449] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:12.496726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.497907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.498381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.499728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 702/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:12.505026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c5a54-dcbc-4836-bd0e-0babc1ec6710 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:12.505472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c5a54-dcbc-4836-bd0e-0babc1ec6710 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:12.505588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c5a54-dcbc-4836-bd0e-0babc1ec6710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:12.574225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4d3c5a54-dcbc-4836-bd0e-0babc1ec6710 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:12.575657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4d3c5a54-dcbc-4836-bd0e-0babc1ec6710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:12.596326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:12.675009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 749/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:12.965634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:12.967987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:13.071238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f3e71-aeee-4576-bb28-5dbb0b5af83d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:13.092709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:13.145405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:13.146007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:13.146282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:13.146554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:13.148422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:13.256405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:13.257773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b6b76-3901-4b77-8ee2-c3800c48256f service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:13.276879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 703/1452] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:15.252777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.253307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.253307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.254688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 750/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:15.259828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e11a-314d-484b-8cd4-4bbefc07e00c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.260249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e11a-314d-484b-8cd4-4bbefc07e00c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.260411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e11a-314d-484b-8cd4-4bbefc07e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.322552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.326003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7a18e11a-314d-484b-8cd4-4bbefc07e00c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:15.327217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7a18e11a-314d-484b-8cd4-4bbefc07e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:15.422499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 704/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:15.708614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:15.710140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:15.813323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95472c-9c4b-4773-a024-5baefccdac3a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:15.862114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:15.886746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:15.887346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:15.888502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:15.888856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:15.890724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:16.004397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:16.005877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764ada4-9001-4e78-a0c3-ae68f2ff5d7c service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:16.024894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 751/1455] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:17.996195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558f05-14bb-4470-bc14-909504c31830 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:17.996876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558f05-14bb-4470-bc14-909504c31830 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:17.996876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558f05-14bb-4470-bc14-909504c31830 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:17.998433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 705/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:18.003849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c21e-7631-492d-8497-17e8ffd71dbc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:18.004268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c21e-7631-492d-8497-17e8ffd71dbc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:18.004432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c21e-7631-492d-8497-17e8ffd71dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:18.070762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f590c21e-7631-492d-8497-17e8ffd71dbc None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:18.071887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f590c21e-7631-492d-8497-17e8ffd71dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:18.101599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:18.169457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 752/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:18.456929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29558f05-14bb-4470-bc14-909504c31830 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:18.458491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-29558f05-14bb-4470-bc14-909504c31830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:18.563283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29558f05-14bb-4470-bc14-909504c31830 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:18.593174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:18.639225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:18.639827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:18.640104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:18.640404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:18.641989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:18.751470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:18.752892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8ab3-6a8f-42cf-af3e-c37509b43eaa service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:18.771202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 706/1458] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:20.732603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3715f-3930-46d2-9186-15433720f165 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.733226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3715f-3930-46d2-9186-15433720f165 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.733226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3715f-3930-46d2-9186-15433720f165 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.734371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 753/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:20.739727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f3c3f-83ba-4920-8cfe-036585de9711 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:20.740170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f3c3f-83ba-4920-8cfe-036585de9711 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:20.740304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f3c3f-83ba-4920-8cfe-036585de9711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:20.804567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e67f3c3f-83ba-4920-8cfe-036585de9711 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:20.805763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e67f3c3f-83ba-4920-8cfe-036585de9711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:20.814578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:20.901353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 707/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:21.183979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3715f-3930-46d2-9186-15433720f165 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:21.186047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-95e3715f-3930-46d2-9186-15433720f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:21.290206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3715f-3930-46d2-9186-15433720f165 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:21.344429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:21.363056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:21.363686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:21.363958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:21.364258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:21.365930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:21.471062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:21.472425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cc080-007c-4b8c-a9c5-bbf524b49d54 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:21.490999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 754/1461] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:23.451920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.452523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.452523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.453822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 708/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:23.459157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96770df-57dd-405f-9db8-a8f3a20aaef9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:23.459611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96770df-57dd-405f-9db8-a8f3a20aaef9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:23.459725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96770df-57dd-405f-9db8-a8f3a20aaef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:23.525912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f96770df-57dd-405f-9db8-a8f3a20aaef9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:23.527134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f96770df-57dd-405f-9db8-a8f3a20aaef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:23.566407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:23.625191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 755/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:23.907385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:23.908983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:24.011867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478511ae-703a-44c0-8a95-36ad9d93c235 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:24.049223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:24.085728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:24.086321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:24.086595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:24.086861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:24.088447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:24.194718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:24.196080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bb369-5fe0-447c-b400-186714739b70 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:24.215585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 709/1464] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:26.163488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.164020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.164020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.165405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 756/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:26.171295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640656-b942-41f3-8e7a-d4fdc76836fa None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.171730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640656-b942-41f3-8e7a-d4fdc76836fa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.171839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640656-b942-41f3-8e7a-d4fdc76836fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.236220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0a640656-b942-41f3-8e7a-d4fdc76836fa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:26.237425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0a640656-b942-41f3-8e7a-d4fdc76836fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:26.254839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.333180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 710/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:26.622935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:26.624486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:26.730369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1043f-1b11-4015-9ba6-0e59c9640632 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:26.797965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:26.804931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:26.805531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:26.805811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:26.806085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:26.807661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:26.928418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:26.929800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5f29e-7c90-4ef0-b4e1-84d5cf96d807 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:26.947847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 757/1467] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:28.880827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.881437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.881437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.882740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 711/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:28.888497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04397b4-fe68-42b8-aaac-9acb053dd6ed None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:28.888968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04397b4-fe68-42b8-aaac-9acb053dd6ed None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:28.889083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04397b4-fe68-42b8-aaac-9acb053dd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:28.958093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d04397b4-fe68-42b8-aaac-9acb053dd6ed None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:28.959311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d04397b4-fe68-42b8-aaac-9acb053dd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:29.007883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:29.056825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 758/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:29.344925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:29.346537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:29.448112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89629457-289c-442c-93e9-3a7327ce4739 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:29.496019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:29.522870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:29.523493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:29.523751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:29.524023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:29.525644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:29.635417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:29.639192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc4d15-8d6c-40f7-9327-e6db21c3a66e service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:29.656702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 712/1470] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:39.054951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=22 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:39.654413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=66 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.306159 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc268e-04d6-4794-94a0-865fcf8486d5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.306678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc268e-04d6-4794-94a0-865fcf8486d5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.306678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc268e-04d6-4794-94a0-865fcf8486d5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.308694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 759/1471] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:49.314290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43335ed-bcf4-42af-bf79-c7048c914b3c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.314716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43335ed-bcf4-42af-bf79-c7048c914b3c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.314848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43335ed-bcf4-42af-bf79-c7048c914b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.380939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b43335ed-bcf4-42af-bf79-c7048c914b3c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:49.382141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b43335ed-bcf4-42af-bf79-c7048c914b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:49.475487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 713/1472] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:49.773419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.773701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.773969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.775305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 760/1473] 10.222.0.75 () {54 vars in 967 bytes} [Thu Aug 25 03:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:49.782029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fff19-b58a-4f55-b084-f929c202491a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:49.782492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fff19-b58a-4f55-b084-f929c202491a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:49.782770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fff19-b58a-4f55-b084-f929c202491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:49.858710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-766fff19-b58a-4f55-b084-f929c202491a None None] MFA Rules not processed for user `ca96b68db5ab4c42ae4d34f6ce0a21d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:49.859928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-766fff19-b58a-4f55-b084-f929c202491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:49.880753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:49.938160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 714/1474] 10.222.0.75 () {58 vars in 1064 bytes} [Thu Aug 25 03:00:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:49.946002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:49.947546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:50.054422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce54ff-f229-4918-87c2-abe1948fec07 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:50.131267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:50.131865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:50.132139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:50.132433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:50.134004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:50.187135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:50.241772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:50.243190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c442-fde8-45df-a256-5b19879e29d1 service glance] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:50.261335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 761/1475] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:52.266603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:52.267208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:52.267315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:52.268718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 715/1476] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:52.275339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ed3a-3c74-4fae-93ce-1860c85cfa08 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:52.275812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ed3a-3c74-4fae-93ce-1860c85cfa08 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:52.275929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ed3a-3c74-4fae-93ce-1860c85cfa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:52.346672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-92c9ed3a-3c74-4fae-93ce-1860c85cfa08 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:52.347884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-92c9ed3a-3c74-4fae-93ce-1860c85cfa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:52.455394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 762/1477] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:00:52.744698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:52.746812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:52.882627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b5925-3128-41cf-a2ad-dcc11cbf4101 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:52.933692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:52.977838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:52.978441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:52.978720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:52.978994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:52.979632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:52.981953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:00:52.981953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:00:52.981953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:00:52.983103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4bb0c-6f89-4950-846d-636c4317301b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:53.001900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 716/1478] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Aug 25 03:00:52 2022] GET /identity/v3/projects => generated 2553 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:00:54.938633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.939060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.939188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:54.940590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 763/1479] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:54.945832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55259fd-794c-4280-9746-d395b1530fac None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:54.946240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55259fd-794c-4280-9746-d395b1530fac None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:54.946372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55259fd-794c-4280-9746-d395b1530fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.012490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e55259fd-794c-4280-9746-d395b1530fac None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:55.013710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e55259fd-794c-4280-9746-d395b1530fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:55.110136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 717/1480] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:55.406867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:55.408467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:55.419542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.677233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859c80d-5b90-4343-9b78-0709929e87a2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:55.769522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:55.802540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:55.803135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:55.803410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:55.803672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:55.805269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:55.891633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:55.893238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae96a42-5e23-407b-9516-9f3f1513fb53 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:55.910846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 764/1481] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:00:57.999573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.000092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.000234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.001544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 718/1482] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:00:58.006909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d3cb-dcdb-409c-b5f1-5f7596b6dc28 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.007294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d3cb-dcdb-409c-b5f1-5f7596b6dc28 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.007454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0d3cb-dcdb-409c-b5f1-5f7596b6dc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.075505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d5c0d3cb-dcdb-409c-b5f1-5f7596b6dc28 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:00:58.076755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d5c0d3cb-dcdb-409c-b5f1-5f7596b6dc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:58.126903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.178773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 765/1483] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:00:58.470610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:00:58.472224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:58.496018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.576506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d719b2a-6c5b-462c-a246-cfb1d6b82952 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:00:58.650137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:00:58.650769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:00:58.651013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:00:58.651364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:00:58.652956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:00:58.726832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:00:58.762188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:00:58.763636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260567ef-a7d4-418e-8108-ca013badcf01 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:00:58.782646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 719/1484] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:01:00.813118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.813693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.813693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.815022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 766/1485] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:00.820361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4effb-e68e-448f-bc90-85666d9403c1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:00.820819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4effb-e68e-448f-bc90-85666d9403c1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:00.820931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4effb-e68e-448f-bc90-85666d9403c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:00.891875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f5a4effb-e68e-448f-bc90-85666d9403c1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:00.893120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f5a4effb-e68e-448f-bc90-85666d9403c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:00.990407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 720/1486] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:01:01.293409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:01.294948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:01.400384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fce375-f667-4abf-bbd9-bf237359c2ae None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:01.472992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:01.474739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:01.475337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:01.475615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:01.475883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:01.477480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:01.588370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:01.590392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d11cf9-1846-42f2-8316-f92943e9ccfa service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:01.608346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 767/1487] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:01:03.678966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.679510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.679510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.680879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 721/1488] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:03.686631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502f66-94da-4797-9442-7e43f01126f2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:03.687043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502f66-94da-4797-9442-7e43f01126f2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:03.687177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502f66-94da-4797-9442-7e43f01126f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:03.752129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0f502f66-94da-4797-9442-7e43f01126f2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:03.753369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0f502f66-94da-4797-9442-7e43f01126f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:03.805143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:03.849331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 768/1489] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:01:04.143861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:04.145782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:04.181156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:04.249164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e32b70-8225-471e-8604-134cc7e7d468 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:04.321216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:04.321827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:04.322107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:04.322381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:04.324584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:04.405703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:04.436831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:04.438203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8c2d-a45e-46e1-ae4d-71972debaad4 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:04.456250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 722/1490] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:01:06.958867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.959346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.959471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:06.960805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 769/1491] 10.222.0.75 () {54 vars in 927 bytes} [Thu Aug 25 03:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:01:06.966650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af4a-99fb-4b5b-8022-ca979d99ecb9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:06.967088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af4a-99fb-4b5b-8022-ca979d99ecb9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:06.967201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af4a-99fb-4b5b-8022-ca979d99ecb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.033300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3581af4a-99fb-4b5b-8022-ca979d99ecb9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:01:07.034505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3581af4a-99fb-4b5b-8022-ca979d99ecb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:07.129791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 723/1492] 10.222.0.75 () {58 vars in 1024 bytes} [Thu Aug 25 03:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:01:07.422855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:01:07.424408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:07.529454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd1f1e-5958-46c6-9197-6dc85f472ca4 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:01:07.601741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:01:07.602340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:01:07.602614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:01:07.602886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:01:07.604501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:01:07.614493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:07.713620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:01:07.715187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce324f00-9c9f-49c2-a365-dfaab7ff31ce service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:01:07.733487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 770/1493] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:01:17.127973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=82 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:01:17.731260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=50 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:28.711787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:28.712431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:28.712431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:28.787652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:28.788894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:28.885420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 724/1494] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:02:28.893712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:28.894208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:28.894334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:28.976731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:28.977855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:29.073048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 771/1495] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:02:29.077795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:29.079016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:29.091188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9b9b-d2ef-4225-9c07-927518bef388 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:29.152510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:29.185600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:29.186205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:29.186487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:29.186762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:29.187459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:02:29.187781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:29.189466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:29.189466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:02:29.189466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:02:29.190794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:29.202113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 725/1496] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:02:29.210405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:29.220180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edb43-d67a-4233-a979-fe871cf23094 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:29.280735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:29.313737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:29.314342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:29.314625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:29.314917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:29.315532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:29.317341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:29.317341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:02:29.317341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:02:29.318560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:29.465745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 772/1497] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:02:29 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:29.470027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:29.471614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:29.506410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af914-1962-4ced-a4a1-00e1156a5eb2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:29.599229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:29.599749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:29.600051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:29.600357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:29.600999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:29.602089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:29.602089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:02:29.602089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:02:29.603303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:29.726753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 726/1498] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:02:29 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:02:29.731147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:29.732678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:29.794681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb027d-9e13-45ff-8d39-b1da4a52a532 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:29.877017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:29.887740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:29.888462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:29.888734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:29.889001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:29.889639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:29.890733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:29.890733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:02:29.890733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:02:29.891608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:29.905080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 773/1499] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:02:29.908924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:29.910491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:29.922479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9186fc9-a3ae-4959-b518-695eaa0c0eb4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:29.949977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.027982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.028698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.028972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.029242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.029881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:30.030895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:30.030895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:02:30.030895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:02:30.031784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c57da4-db01-4c67-943d-38cf143aa41e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:30.045582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 727/1500] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:02:30.049772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.051319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.085119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fec238-110b-452d-b3de-55e4d6c7762c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:30.179720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.180327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.180606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.180876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] PATH_INFO: `/v3/projects/878245276275421988f047dda9bcc834/users/5cc7119ae23d44aba3cc6dacabe5b300/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.203534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=878245276275421988f047dda9bcc834, user_id=5cc7119ae23d44aba3cc6dacabe5b300, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:30.204603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:30.204603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:02:30.204603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:02:30.206547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86dfa-acba-45e4-9a71-3c088ef90bdd admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:30.283694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 774/1501] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:02:30 2022] PUT /identity/v3/projects/878245276275421988f047dda9bcc834/users/5cc7119ae23d44aba3cc6dacabe5b300/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:30.296384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da396-1718-4d43-8e76-f3cf2064f720 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.296670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da396-1718-4d43-8e76-f3cf2064f720 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.296940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da396-1718-4d43-8e76-f3cf2064f720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.352446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-490da396-1718-4d43-8e76-f3cf2064f720 None None] MFA Rules not processed for user `5cc7119ae23d44aba3cc6dacabe5b300`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:30.354460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-490da396-1718-4d43-8e76-f3cf2064f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.431461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 728/1502] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:02:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:02:30.444909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.445185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.445461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.446785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 775/1503] 10.222.0.75 () {54 vars in 964 bytes} [Thu Aug 25 03:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:30.451730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97b39-ed46-47ad-a7d2-b93dac7c3271 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.452013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97b39-ed46-47ad-a7d2-b93dac7c3271 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.452312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97b39-ed46-47ad-a7d2-b93dac7c3271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.483151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.526964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6fc97b39-ed46-47ad-a7d2-b93dac7c3271 None None] MFA Rules not processed for user `57f08a89a0dd4befb153c2b38167b5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:30.528218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6fc97b39-ed46-47ad-a7d2-b93dac7c3271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.602621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 729/1504] 10.222.0.75 () {58 vars in 1061 bytes} [Thu Aug 25 03:02:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:02:30.611241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.612742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.663441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.716021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c12fc1-44ff-4ac4-9db5-517d4d0dae10 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:30.788166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:30.788782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.789061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.789336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.791468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:30.888040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:30.889479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:30.893155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:30.908855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 776/1505] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:02:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:02:30.954906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6ad3-aacf-4bd3-b03f-cdf286d41cbf None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:30.955351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6ad3-aacf-4bd3-b03f-cdf286d41cbf None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:30.955455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6ad3-aacf-4bd3-b03f-cdf286d41cbf None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:30.956751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 730/1506] 10.222.0.75 () {52 vars in 830 bytes} [Thu Aug 25 03:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:02:30.960363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:30.961838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:31.070346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cab43-3d7d-4ed6-a460-cd06030cc689 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:31.163618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:31.164261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.164535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.164797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] PATH_INFO: `/v3/users/5cc7119ae23d44aba3cc6dacabe5b300` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.176899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc7119ae23d44aba3cc6dacabe5b300)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.177902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:31.177902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:02:31.177902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:02:31.179088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:31.188671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.414745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc7119ae23d44aba3cc6dacabe5b300'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:31.415287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc7119ae23d44aba3cc6dacabe5b300'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:31.430507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc7119ae23d44aba3cc6dacabe5b300'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:31.508392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc7119ae23d44aba3cc6dacabe5b300'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:31.508849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc7119ae23d44aba3cc6dacabe5b300'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:31.538577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 777/1507] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:02:30 2022] DELETE /identity/v3/users/5cc7119ae23d44aba3cc6dacabe5b300 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:31.543136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.547397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.547397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.600712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:02:31.601915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:31.678442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.691410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 731/1508] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:02:31.701851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:31.703219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:31.808537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28d05c-9439-4dfb-80ac-7c07802f5d9e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:31.860377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:31.879764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:31.880519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:31.880794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:31.881061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:31.882659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:31.992093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:31.993533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d9bda-0f2f-4351-a4fc-612fa48cb0df service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:32.013586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 778/1509] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:02:32.499584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:02:32.500902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:02:32.513141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52945c37-4c70-4a4b-8b5a-3b98ee5cca48 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:02:32.611427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:02:32.612061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:02:32.612364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:02:32.612637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] PATH_INFO: `/v3/projects/878245276275421988f047dda9bcc834` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:02:32.622512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878245276275421988f047dda9bcc834)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:02:32.623542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:02:32.623542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:02:32.623542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:02:32.624835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:02:32.740434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878245276275421988f047dda9bcc834', 'request_id': 'req-3f5be10b-749e-43b2-acbb-445fb78ca968'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:32.746647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878245276275421988f047dda9bcc834', 'request_id': 'req-3f5be10b-749e-43b2-acbb-445fb78ca968'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:32.838408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878245276275421988f047dda9bcc834', 'request_id': 'req-3f5be10b-749e-43b2-acbb-445fb78ca968'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:32.870032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] The token cache is being invalidate because project 878245276275421988f047dda9bcc834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:02:32.870315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:32.870839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-3f5be10b-749e-43b2-acbb-445fb78ca968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:02:32.872220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 732/1510] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:02:32 2022] DELETE /identity/v3/projects/878245276275421988f047dda9bcc834 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:02:42.011302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=62 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:02:42.867309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:05.849738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:05.850343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:05.850343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:05.851710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 779/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:05.856538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a6481-b341-4c03-9b3c-8bfeb913bc01 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:05.856970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a6481-b341-4c03-9b3c-8bfeb913bc01 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:05.857075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a6481-b341-4c03-9b3c-8bfeb913bc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:05.959600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e27a6481-b341-4c03-9b3c-8bfeb913bc01 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:05.959600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e27a6481-b341-4c03-9b3c-8bfeb913bc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:06.056463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 733/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:06.211851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:06.213975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:06.333382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024bb2-2aa2-4d4c-90a4-66425539d0dc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:06.338777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:06.339384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:06.339661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:06.339932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:06.348781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:06.350361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eedc5e-d884-4606-9f4f-caeb4f785fea service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:06.359692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 780/1513] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:07.237990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f615c-b1c3-4b6c-aeae-8ad76acbc088 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:07.238542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f615c-b1c3-4b6c-aeae-8ad76acbc088 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:07.238542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f615c-b1c3-4b6c-aeae-8ad76acbc088 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:07.239810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 734/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:04:07.244656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc58b-b74c-4ca9-bce2-c762f202d19a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:07.245085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc58b-b74c-4ca9-bce2-c762f202d19a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:07.245192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc58b-b74c-4ca9-bce2-c762f202d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:07.265932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1e3bc58b-b74c-4ca9-bce2-c762f202d19a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:07.267133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1e3bc58b-b74c-4ca9-bce2-c762f202d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:07.275102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 781/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:07.440793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f615c-b1c3-4b6c-aeae-8ad76acbc088 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:07.455665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f615c-b1c3-4b6c-aeae-8ad76acbc088 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:07.461431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:07.462037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:07.462339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:07.462641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:07.471755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:07.476575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8b1f1-8642-4a24-b3e6-698978f3023d service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:07.482910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 735/1516] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:08.165330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf038c-01cd-4e2a-9908-7563a169878d None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:08.165611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf038c-01cd-4e2a-9908-7563a169878d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:08.165883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf038c-01cd-4e2a-9908-7563a169878d None None] PATH_INFO: `/v3` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:08.167166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 782/1517] 10.222.0.79 () {54 vars in 925 bytes} [Thu Aug 25 03:04:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:04:08.173467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784cf6-135d-4633-8782-9c3e84c62918 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:08.173761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784cf6-135d-4633-8782-9c3e84c62918 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:08.174038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784cf6-135d-4633-8782-9c3e84c62918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:08.233636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c2784cf6-135d-4633-8782-9c3e84c62918 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:08.234857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c2784cf6-135d-4633-8782-9c3e84c62918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:08.251966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:08.306575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 736/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Thu Aug 25 03:04:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:12.405097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b604-cf1f-45ed-87ce-f8ecd25945b7 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:12.405658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b604-cf1f-45ed-87ce-f8ecd25945b7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:12.405658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b604-cf1f-45ed-87ce-f8ecd25945b7 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:12.406971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 783/1519] 10.222.0.75 () {54 vars in 961 bytes} [Thu Aug 25 03:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:12.411802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aefd2-283d-4af5-be80-32d30c2eb435 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:12.412222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aefd2-283d-4af5-be80-32d30c2eb435 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:12.412391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aefd2-283d-4af5-be80-32d30c2eb435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:12.431218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-871aefd2-283d-4af5-be80-32d30c2eb435 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:12.432556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-871aefd2-283d-4af5-be80-32d30c2eb435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:12.441465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 737/1520] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Aug 25 03:04:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:12.448770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11b604-cf1f-45ed-87ce-f8ecd25945b7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:12.462683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11b604-cf1f-45ed-87ce-f8ecd25945b7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:12.468093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:12.468719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:12.468998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:12.469271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:12.477762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:12.479653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703b2b8-727b-4ea5-b982-a920f29d4c11 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:12.489621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 784/1521] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:18.723976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676483f6-940e-441d-88a0-36d575ec0f3a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.724544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676483f6-940e-441d-88a0-36d575ec0f3a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.724544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676483f6-940e-441d-88a0-36d575ec0f3a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.725905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 738/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:18.730870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a530f3-2182-4f5d-b9e7-4e9f21817164 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.731293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a530f3-2182-4f5d-b9e7-4e9f21817164 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.731420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a530f3-2182-4f5d-b9e7-4e9f21817164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.754444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-56a530f3-2182-4f5d-b9e7-4e9f21817164 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:18.755635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-56a530f3-2182-4f5d-b9e7-4e9f21817164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:18.763548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 785/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:18.916914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676483f6-940e-441d-88a0-36d575ec0f3a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:18.933817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676483f6-940e-441d-88a0-36d575ec0f3a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:18.938693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:18.939294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:18.939575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:18.939845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:18.948512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:18.950196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5268ea-6cd4-4afa-bfd6-39bebcac4d93 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:18.959881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 739/1524] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:19.787191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914b9e-ab29-4ae6-a10b-1ff9913b914b None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.787763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914b9e-ab29-4ae6-a10b-1ff9913b914b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.787763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914b9e-ab29-4ae6-a10b-1ff9913b914b None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:19.789136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 786/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 25 03:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:19.793997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86ae5-99ca-465d-ab22-be01523f16b6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.794559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86ae5-99ca-465d-ab22-be01523f16b6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.794658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86ae5-99ca-465d-ab22-be01523f16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:19.814706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-dcd86ae5-99ca-465d-ab22-be01523f16b6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:19.815902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-dcd86ae5-99ca-465d-ab22-be01523f16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:19.823761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 740/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 25 03:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:19.976501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914b9e-ab29-4ae6-a10b-1ff9913b914b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:19.991452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914b9e-ab29-4ae6-a10b-1ff9913b914b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:19.996903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:19.998161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:19.998484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:19.998671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:20.006911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:20.008516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ca93-515d-447b-8511-6bb061d4f91d service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:20.018770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 787/1527] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:24.535820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:24.536403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:24.536403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:24.537812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 741/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:24.542930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d6bb-23c5-47ed-8614-9778e0ce18f0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:24.543354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d6bb-23c5-47ed-8614-9778e0ce18f0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:24.543452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d6bb-23c5-47ed-8614-9778e0ce18f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:24.563377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ba15d6bb-23c5-47ed-8614-9778e0ce18f0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:24.564614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ba15d6bb-23c5-47ed-8614-9778e0ce18f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:24.573003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 788/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:24.863032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:24.864820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:24.945763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34da1b6-88a7-411a-bd0f-2b9b40a24219 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:24.949293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:24.949893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:24.952064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:24.952389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:24.961582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:24.963014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b15b39-b949-43d8-b82c-ea17cd3287c4 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:24.979987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 742/1530] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:26.867705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511272-b45a-43a7-b67c-d484f1792839 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.868267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511272-b45a-43a7-b67c-d484f1792839 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.868267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511272-b45a-43a7-b67c-d484f1792839 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.869650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 789/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:26.874851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cef8-c9ca-4206-8355-19c008650803 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:26.875297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cef8-c9ca-4206-8355-19c008650803 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:26.875396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90cef8-c9ca-4206-8355-19c008650803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:26.894679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-fe90cef8-c9ca-4206-8355-19c008650803 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:26.895887 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fe90cef8-c9ca-4206-8355-19c008650803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:26.904857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 743/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:27.194175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f511272-b45a-43a7-b67c-d484f1792839 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:27.206519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f511272-b45a-43a7-b67c-d484f1792839 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:27.210123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:27.210716 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:27.210994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:27.211265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:27.222438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:27.223857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba6698-e615-4d92-8853-2e8cbd464afd service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:27.233349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 790/1533] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:04:30.729829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.730373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.730373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.731864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 744/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:30.736847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa8fc-16ec-4a23-a0fc-ae6812718746 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:30.737308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa8fc-16ec-4a23-a0fc-ae6812718746 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:30.737406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa8fc-16ec-4a23-a0fc-ae6812718746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:30.756677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2cafa8fc-16ec-4a23-a0fc-ae6812718746 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:30.757865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2cafa8fc-16ec-4a23-a0fc-ae6812718746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:30.766190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 791/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:31.317920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:31.319584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:31.383835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:31.397522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ce537-06d5-4569-9558-4a89cee68413 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:31.401156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:31.401766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:31.402045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:31.402314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:31.411276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:31.412752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbd068-19a0-48b1-a1a7-59f566b4f441 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:31.422072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 745/1536] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:33.345475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:33.346025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:33.346025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:33.347448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 792/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 25 03:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:04:33.352622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efe17-771f-4026-9f3a-7cea6f650ede None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:33.353024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efe17-771f-4026-9f3a-7cea6f650ede None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:33.353174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660efe17-771f-4026-9f3a-7cea6f650ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:33.372955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-660efe17-771f-4026-9f3a-7cea6f650ede None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:04:33.374156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-660efe17-771f-4026-9f3a-7cea6f650ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:33.381934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 746/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 25 03:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:04:33.929271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:04:33.931135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:04:33.938781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=84 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:33.944686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29010ee2-0fb6-446f-bcf9-cd1614c3fbfd None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:04:33.948048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:04:33.948669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:04:33.948968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:04:33.949235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:04:33.958731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:04:33.960265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95163b65-691a-4c74-91e8-b3b974d62ed2 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:04:33.970490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 793/1539] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:04:43.368307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=11 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:04:43.983795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:05:58.381097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8a81-91af-43f6-a0a6-54c8944846e6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.387726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8a81-91af-43f6-a0a6-54c8944846e6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.387726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d8a81-91af-43f6-a0a6-54c8944846e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.409426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-de5d8a81-91af-43f6-a0a6-54c8944846e6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.410669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-de5d8a81-91af-43f6-a0a6-54c8944846e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.427073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 747/1540] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:58.434944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.436038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.436535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.475777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.477767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.482334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8efc1-b6b5-44c4-9bec-bcab39f4a15d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.487570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8efc1-b6b5-44c4-9bec-bcab39f4a15d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.487570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8efc1-b6b5-44c4-9bec-bcab39f4a15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.511317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 794/1541] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:58.512991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:58.517631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1ad8efc1-b6b5-44c4-9bec-bcab39f4a15d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.519263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1ad8efc1-b6b5-44c4-9bec-bcab39f4a15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.531123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fb928-d308-466a-958f-a9927ebcb287 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:58.535834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 748/1542] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:05:58.546203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.546440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:58.546843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.547131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.547306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.547431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.547847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.548051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:58.548375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:58.549600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:05:58.549600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:05:58.549600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:05:58.550887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:58.581002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.582506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.582766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 795/1543] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:58.592386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:58.596411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 749/1544] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:05:58.601336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:58.612115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40f91e-40d4-4380-9c2a-a915c2e3f2ea None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:58.616385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b3598-c597-4b0a-82d4-412cdf690a70 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:58.622386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:58.623477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.624065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.624561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.625539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:58.626045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:58.632242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:58.632242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.632242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.632242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.632650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:58.633183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:58.638275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab34994-b9b9-4180-947e-e7e39559477a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:58.651261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 750/1545] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:58.659620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:58.677092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbe413-e89e-47af-bb39-f91c9d202081 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:58.691305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:58.692292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.692882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.693354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.694195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:58.695413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:05:58.695413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:05:58.695413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:05:58.697109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441db66-4450-49d3-ab47-11fd8e8e4275 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:58.798093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 796/1546] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:58.799784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902f0a-1d69-4dee-b581-37948b2dd30a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.800411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902f0a-1d69-4dee-b581-37948b2dd30a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.800927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902f0a-1d69-4dee-b581-37948b2dd30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.823027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9b902f0a-1d69-4dee-b581-37948b2dd30a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.824328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9b902f0a-1d69-4dee-b581-37948b2dd30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.887874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 751/1547] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:05:58.889661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e96d-ac50-4cd5-82b9-a54f784891ce None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.889800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 797/1548] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:58.889897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e96d-ac50-4cd5-82b9-a54f784891ce None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.889897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e96d-ac50-4cd5-82b9-a54f784891ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.890664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.891043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.891518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.912442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2197e96d-ac50-4cd5-82b9-a54f784891ce None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.912683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:58.913686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.913905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2197e96d-ac50-4cd5-82b9-a54f784891ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:58.977126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 798/1549] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:58.977957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:58.979821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 752/1550] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:58.981508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ac86-9278-480c-8be5-7f977754d01d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.981992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ac86-9278-480c-8be5-7f977754d01d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.982541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ac86-9278-480c-8be5-7f977754d01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.992175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85df23-a091-4244-b4ab-f4ed88e68c60 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:58.997220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:58.997829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:58.998112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:58.998382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:58.999125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.000167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:05:59.000167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:05:59.000167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:05:59.001359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a52f53-d2a5-4f16-9574-460db8b45bc7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.003074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-af83ac86-9278-480c-8be5-7f977754d01d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.004302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-af83ac86-9278-480c-8be5-7f977754d01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.012928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 753/1551] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.014132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4bf3e-ced8-409a-ab96-3aa25d70a6af None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.014411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4bf3e-ced8-409a-ab96-3aa25d70a6af None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.014680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4bf3e-ced8-409a-ab96-3aa25d70a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.034194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3ff4bf3e-ced8-409a-ab96-3aa25d70a6af None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.035386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3ff4bf3e-ced8-409a-ab96-3aa25d70a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.043841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 754/1552] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.045023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.045330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.045602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.063864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.065046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.073072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 755/1553] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.073634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.082642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42ddec-722b-480c-b98e-9f0d33580be3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.094856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.094856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.094856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.094856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.095307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.095307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30413a7f-ddf8-4092-bd4a-57d7a8611be1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.205903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 799/1554] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:05:58 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.207177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc022-18a8-4f0a-b799-753d3a6ad0be None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.207459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc022-18a8-4f0a-b799-753d3a6ad0be None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.207728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc022-18a8-4f0a-b799-753d3a6ad0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.227486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-425bc022-18a8-4f0a-b799-753d3a6ad0be None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.228986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-425bc022-18a8-4f0a-b799-753d3a6ad0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.236518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 800/1555] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.237648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ad87-5abe-4c22-95ff-d4235544c198 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.237966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ad87-5abe-4c22-95ff-d4235544c198 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.238234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ad87-5abe-4c22-95ff-d4235544c198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.252865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 756/1556] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.268813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff46461-bb10-4366-885d-72bd264f3f65 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.268813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff46461-bb10-4366-885d-72bd264f3f65 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.268813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff46461-bb10-4366-885d-72bd264f3f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9102ad87-5abe-4c22-95ff-d4235544c198 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9102ad87-5abe-4c22-95ff-d4235544c198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 801/1557] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad001b-4e44-429f-8cfd-aa17557ea432 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad001b-4e44-429f-8cfd-aa17557ea432 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.269178 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad001b-4e44-429f-8cfd-aa17557ea432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.277677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4ff46461-bb10-4366-885d-72bd264f3f65 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.278915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4ff46461-bb10-4366-885d-72bd264f3f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.287343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 757/1558] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.288571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.288944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.289123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.289766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-93ad001b-4e44-429f-8cfd-aa17557ea432 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.291204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-93ad001b-4e44-429f-8cfd-aa17557ea432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.299154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 802/1559] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.300441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.300730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.301063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.309600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.310801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.323142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 758/1560] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.323142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.323631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:05:59.323731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:05:59.330558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029852d-fb1e-488e-afdb-72a361d4132f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.331830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 803/1561] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:05:59.335580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.335674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.335814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.336181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.336453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.337097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.339969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.342867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e8673-88ab-442e-9967-2fa4191985db None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.347923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.348541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.348819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.349094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.349736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.352978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 759/1562] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:05:59.353114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.353234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.353333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.353594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:05:59.360753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0843f5-b6b6-4be1-b8fa-61fb6d3a1a38 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.387635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 804/1563] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.387635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.387895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.387895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.387895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.387895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.388692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.388692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.388871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e0204-6c60-4ba7-948b-1d5dda342896 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.388871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.389060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 760/1564] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:05:59.389147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.389147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.389147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.389397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.389488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.389488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.389488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.395438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 805/1565] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.396268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.397754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582622a4-f1f4-4dc3-9429-16be6ac0b4d0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.405018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.405644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.405927 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.406201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.406852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.407029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f3cb-783f-4b0d-bd50-e9e29511a472 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.407155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.408794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.411608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.412230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.412515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.412789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.413431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.413733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.415315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.422162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 761/1566] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.422995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.428752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 806/1567] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.429593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.435872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceeff2-a4a8-4701-b558-986e27141b09 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.461470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b673d-a46b-421d-a3a6-68ac49d138e8 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.468582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.468582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.468582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.468582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.468974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.468974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.469144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.469144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.469342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.469439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.469439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.469439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:05:59.469439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.469439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.469827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 762/1568] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.469827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.469979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 807/1569] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:05:59.469979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.470128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8db5-0d3c-4e1c-bc63-7f4461d64a5e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.474262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.475305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.475305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.475598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.476057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.477706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.479226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:05:59.479639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e07b04-6f8d-46fa-a3b2-a8bfe2ce4023 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.484224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.484828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.485109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.485413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.486061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.487941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.488656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 763/1570] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:05:59.489505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.499901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde9e62-9e1b-419a-b91d-9f50da29e917 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.505108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.505713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.505992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.506261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.506953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.508616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.519820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 764/1571] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:05:59.520645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.530499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed363f-ad2b-485b-91d8-8c8a228a58a2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.534955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.535510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.535783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.536050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.536677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.538521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.589344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 808/1572] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.590089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.652736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af80a9-b91f-4d33-a02a-5d249214d5f4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.657409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.662267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.662267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.662267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.662267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.662267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.663898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 765/1573] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.664921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.726065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a4d60-7c2c-4950-b112-bb31dce05626 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.730754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.731417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.731693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.731962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.732613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.734519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.761261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 809/1574] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.762184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.819375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 766/1575] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:05:59.819375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:05:59.822840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f633ef-3fd7-4c09-b25d-579f75b4d5da admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.842628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:05:59.879392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.880000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.880331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.880606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.880994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca9370-7b87-451f-adce-dcce6a896c1d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:05:59.881228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.883207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:05:59.886596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:05:59.887215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:05:59.887492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:05:59.887761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:05:59.888412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:05:59.890300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.055207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 810/1576] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 373 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.055207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.099065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 767/1577] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:05:59 2022] POST /identity/v3/projects => generated 375 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.099882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.118663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb4a2-85f9-4faa-82b1-f8facb8f30bd admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.135436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:00.165107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7efdf-af1e-478c-b8ce-5b0f0c31841d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.174760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.174760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.174760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.174760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] PATH_INFO: `/v3/projects/1d3d0ec7db3c4b3b8625c575a2a11ec8/users/fe93db6503a143ab9ed50fedbc8f010f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.176718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.177377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.177655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.177923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] PATH_INFO: `/v3/projects/af2fb18a62fc4528955820da1bb6a409/users/4b694ae9029848f683d475646e7cd589/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.187576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3d0ec7db3c4b3b8625c575a2a11ec8, user_id=fe93db6503a143ab9ed50fedbc8f010f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.188678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:06:00.188678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:06:00.188678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:06:00.190716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.192858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2fb18a62fc4528955820da1bb6a409, user_id=4b694ae9029848f683d475646e7cd589, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.195468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.319089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 768/1578] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:00 2022] PUT /identity/v3/projects/1d3d0ec7db3c4b3b8625c575a2a11ec8/users/fe93db6503a143ab9ed50fedbc8f010f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:00.320020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.379781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c920a7-e3be-44cf-baa2-9217ad12ead1 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.386571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 811/1579] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:00 2022] PUT /identity/v3/projects/af2fb18a62fc4528955820da1bb6a409/users/4b694ae9029848f683d475646e7cd589/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:00.386571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.437580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.450640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.450640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.450640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.450640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.451059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:00.451168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.459114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce656f-3c84-4bd3-bd8a-03d3d6ddd68d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.464382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.464969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.465251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.465520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.466135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.468056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.559615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 769/1580] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.560635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.570050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dcc4f-3981-4687-a926-02e35f6999ad admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.575113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.575686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.575962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.576289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.576898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.578809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.594493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 812/1581] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 303 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.604397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.605797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d99ce8-4b97-4bcb-9da2-20183fe205c1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.610313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.610886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.611166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.611435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.612040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.613965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.679600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 770/1582] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.680557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.690955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd5ba-0b92-42ed-975a-ee9b6d684b21 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.695546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.696120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.696428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.696701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.697366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.699587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3617-6f15-4c60-89c1-20f37deeb8fa admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.706865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 813/1583] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.709097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.718291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cea610-e8bc-4268-ad7e-7a0e56d291f2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.723040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.723619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.723906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.724192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.724810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.726701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.805664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 771/1584] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.807628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f02e6-fae4-455a-a5fb-992e8ed06745 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.807912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f02e6-fae4-455a-a5fb-992e8ed06745 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.808212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f02e6-fae4-455a-a5fb-992e8ed06745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.826412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-867f02e6-fae4-455a-a5fb-992e8ed06745 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:00.827554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-867f02e6-fae4-455a-a5fb-992e8ed06745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:00.835760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 772/1585] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:00.837206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.837494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.837801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.837983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 814/1586] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:00.838763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.852318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7352-ac68-4026-b39d-9cf0d2bc2f2a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.857985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.858678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.858976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.859250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.859953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.861580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.862154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:00.870827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] MFA Rules not processed for user `4b694ae9029848f683d475646e7cd589`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:00.872044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:00.872954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 815/1587] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:00.873853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.885422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52885b1-ffaa-458d-87f2-ba28a89c20a1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.889991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.890572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.890900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.891177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.891804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.893404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.903090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 816/1588] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:00.903863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.914633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3d3a-f224-4359-8234-2f3d36925a09 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.919852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.921374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.921374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.921374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.921685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.923256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.936484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 817/1589] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:00.937319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.946883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1620ea-520e-4c36-b797-fb8ce8d99617 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.947430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 773/1590] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:00.948756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.951860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.952485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.952763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.953030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.953659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.955415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.961134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fa3cc-75ba-4054-aa76-42c0b4528883 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.965695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 818/1591] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:00.966355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.969292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.970024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.970385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.970725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.971601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.973386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:00.976990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a14323-fa70-454c-ac5a-e0e048c74f10 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:00.982710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:00.983315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:00.983593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:00.983945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:00.984613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:00.985112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 774/1592] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:00.985933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:00.989270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.003467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 819/1593] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.003467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.012907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2de2b-165b-45c7-a6ab-9dbef0b0d502 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.018061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.018691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.018974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.019229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.019857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.021465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.032120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 820/1594] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.033175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.043783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abb52e-153e-4020-a30c-13f79c35a62b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.048455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.049045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.049318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.049587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.050224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.051794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.053677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1322-8f36-4696-b83d-b5cb3f0a90f8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.058178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.058780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.059091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.059369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.062966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 821/1595] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.063966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.068989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.070455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.073824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5de-0a04-4d23-aa36-ad9e513f8d51 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.079738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.079738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.079738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.080105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.080589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.081492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 775/1596] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:01.082176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9706b-82f2-427e-a6b0-a8a24592998a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.082505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.093335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 822/1597] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.093971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e09025-3f3f-4de8-9dd7-950e9c7c6da5 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.094598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f76f-1116-42e7-b353-625ec2c662e7 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.094988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f76f-1116-42e7-b353-625ec2c662e7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.095164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4f76f-1116-42e7-b353-625ec2c662e7 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.096793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 823/1598] 10.222.0.75 () {54 vars in 962 bytes} [Thu Aug 25 03:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:01.097563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f76f-1116-42e7-b353-625ec2c662e7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.102067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.102893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.103178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.103518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.105064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.106985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.111577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4f76f-1116-42e7-b353-625ec2c662e7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.116270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.124631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.124631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.124631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.124631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.124973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 776/1599] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.124973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.125120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.129333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd6ec6-355d-410b-bb4f-de99c4b320fc admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.136364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 824/1600] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.136364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.136611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.136611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.136611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.136611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.137122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.138864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.144277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245ef1-b5cf-41d8-9961-f33f1bd344aa admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.149307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.149878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 777/1601] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:01.150010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.150750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.151459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] PATH_INFO: `/v3/projects/df88838ef3784d6883bea9c96c635d05/users/c14aeccc28d3415aa7d21686e6a0dd1a/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.151631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.162699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0ee30-75af-4566-9c36-11b80436f7cc admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.170537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.170537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.170537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.170537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] PATH_INFO: `/v3/projects/bc5b9a0d35974e5e83f4e739558d0a2d/users/72e948509c3444eaacf205434e6db4e4/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.170962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df88838ef3784d6883bea9c96c635d05, user_id=c14aeccc28d3415aa7d21686e6a0dd1a, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.173261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.184270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5b9a0d35974e5e83f4e739558d0a2d, user_id=72e948509c3444eaacf205434e6db4e4, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.186932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0fada-8ec6-4200-b749-c7cac5d1b920 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.308180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 825/1602] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/df88838ef3784d6883bea9c96c635d05/users/c14aeccc28d3415aa7d21686e6a0dd1a/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.309110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.347009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:01.348722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 778/1603] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/bc5b9a0d35974e5e83f4e739558d0a2d/users/72e948509c3444eaacf205434e6db4e4/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.350450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.350877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.351345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.379066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a9333-bc22-4127-8785-f9c02c0cb0cc admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.396220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:01.398037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:01.434059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:01.450431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.451112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.451449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.451711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] PATH_INFO: `/v3/projects/d5a66dfea4c5492da1ac3e1f76cbe35d/users/7daac509316048ec903a5d78d3586aad/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.470189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a66dfea4c5492da1ac3e1f76cbe35d, user_id=7daac509316048ec903a5d78d3586aad, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.473319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.482467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 779/1604] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Aug 25 03:06:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:01.483484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.493788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4916d80-6415-4b74-958d-3351959b7a93 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.498699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.499351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.499630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.499903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] PATH_INFO: `/v3/projects/eeb0f7283fb84ea48fd468f22c721b8b/users/4529d06c4eda48f788ce7c0327366377/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.516527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb0f7283fb84ea48fd468f22c721b8b, user_id=4529d06c4eda48f788ce7c0327366377, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.523231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.582361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 826/1605] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/d5a66dfea4c5492da1ac3e1f76cbe35d/users/7daac509316048ec903a5d78d3586aad/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.583195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.640093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 780/1606] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/eeb0f7283fb84ea48fd468f22c721b8b/users/4529d06c4eda48f788ce7c0327366377/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.641015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.645927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9412-ea5e-439b-bc78-842712178e7a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.712033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a320db-d851-434f-9667-fc4245a0fb08 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:01.715422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.716965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.717242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.717517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] PATH_INFO: `/v3/projects/44ad45bb838d4a27a8c085b28b5a227d/users/94a5fef1b4394d4b81d5ac52e94b6bf2/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.718459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:01.719230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.719577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.719920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] PATH_INFO: `/v3/projects/f626c34f12fb4bdab6bd42b05b79742a/users/804df9333a474723968f97eb9d6bfc69/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.733254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:01.737027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ad45bb838d4a27a8c085b28b5a227d, user_id=94a5fef1b4394d4b81d5ac52e94b6bf2, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.737285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f626c34f12fb4bdab6bd42b05b79742a, user_id=804df9333a474723968f97eb9d6bfc69, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:01.740105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4a50d-1681-4cf4-b6a1-fc4a780c9eef admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.740304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5ed69-6ac0-4ccd-863e-d0053a816dca admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:01.837300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 827/1607] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/44ad45bb838d4a27a8c085b28b5a227d/users/94a5fef1b4394d4b81d5ac52e94b6bf2/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.844534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b475f6-08a3-48a7-a9e4-e4b2bb7774b9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.844534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b475f6-08a3-48a7-a9e4-e4b2bb7774b9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.844534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b475f6-08a3-48a7-a9e4-e4b2bb7774b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.859148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 781/1608] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:01 2022] PUT /identity/v3/projects/f626c34f12fb4bdab6bd42b05b79742a/users/804df9333a474723968f97eb9d6bfc69/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:01.865472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.865756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.870020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.889005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-84b475f6-08a3-48a7-a9e4-e4b2bb7774b9 None None] MFA Rules not processed for user `c14aeccc28d3415aa7d21686e6a0dd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:01.890512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-84b475f6-08a3-48a7-a9e4-e4b2bb7774b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:01.892167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:01.893377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:01.954609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 782/1609] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:01.955976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:01.965200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 828/1610] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:01.966705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465e25-b902-46ad-b06d-1244ec94eeee None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.966990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465e25-b902-46ad-b06d-1244ec94eeee None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.967259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465e25-b902-46ad-b06d-1244ec94eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:01.986757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d1465e25-b902-46ad-b06d-1244ec94eeee None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:01.988007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d1465e25-b902-46ad-b06d-1244ec94eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:01.999747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 829/1611] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:01.999747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf34c-87cc-4858-a4c0-0ef107e0ebde None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:01.999747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf34c-87cc-4858-a4c0-0ef107e0ebde None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:01.999747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf34c-87cc-4858-a4c0-0ef107e0ebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.002994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:02.019676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ecaaf34c-87cc-4858-a4c0-0ef107e0ebde None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:02.021198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ecaaf34c-87cc-4858-a4c0-0ef107e0ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.033906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 830/1612] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:02.033906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f3f3-bcf1-4e71-ba89-9cd8848bd5f8 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.033906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f3f3-bcf1-4e71-ba89-9cd8848bd5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.033906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f3f3-bcf1-4e71-ba89-9cd8848bd5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.062998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f7a7-c76b-4ef4-aa6c-0f15ab8b75f8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.064137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a873f3f3-bcf1-4e71-ba89-9cd8848bd5f8 None None] MFA Rules not processed for user `94a5fef1b4394d4b81d5ac52e94b6bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:02.065438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a873f3f3-bcf1-4e71-ba89-9cd8848bd5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.068875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.069520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.069805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.070080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.135093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.137322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.148288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 831/1613] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:02.148288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.148288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.148288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.152676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 783/1614] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.153592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.155084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.162947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:02.164447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.174581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 832/1615] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:02.175429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.188295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa822bd-3ca0-47ea-9a0c-3efbfa58cf84 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.191529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.192433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.192853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.193219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.193972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.196218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa141d4-e98b-41d3-8c8a-6aec8805f6ea admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.220999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e281f2e-6ee9-4434-af10-710f60bbb504 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.228179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.228179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.228179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.228179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.235843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.237321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.247092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 784/1616] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.247865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.257250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c699a-2404-4345-836d-f91dde34172d service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.262032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.263121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.263121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.263341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.273991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.276120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.286515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 785/1617] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.287326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.297231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e8b5-8a0d-4dd6-84ea-c9021cbb72ef service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.301613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.302137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.302423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.302697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.313853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.315414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4f640-0e31-4a82-a7c8-36b063a0d0d7 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.328939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 786/1618] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.330348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d74ed-6dcf-49e5-941f-84225da65b67 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.330698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d74ed-6dcf-49e5-941f-84225da65b67 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.331085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d74ed-6dcf-49e5-941f-84225da65b67 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.332513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 787/1619] 10.222.0.75 () {54 vars in 962 bytes} [Thu Aug 25 03:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:02.338620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d74ed-6dcf-49e5-941f-84225da65b67 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.366247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 833/1620] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:02.366247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.366247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.366247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.366761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d74ed-6dcf-49e5-941f-84225da65b67 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.386387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:02.387654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.410594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:02.415989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:02.427129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.427992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.428378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.429508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.471311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 834/1621] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:02.473220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.505189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.507532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b1d6-5e42-4043-8078-bfc4ac6fa5cf service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.519924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 788/1622] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.521714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413831-cc79-4353-8f64-23e89a80fdee None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.522026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413831-cc79-4353-8f64-23e89a80fdee None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.522279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413831-cc79-4353-8f64-23e89a80fdee None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.523613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 789/1623] 10.222.0.75 () {54 vars in 912 bytes} [Thu Aug 25 03:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:02.530021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19413831-cc79-4353-8f64-23e89a80fdee None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.544606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19413831-cc79-4353-8f64-23e89a80fdee None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.551995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.552704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.552985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.553258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.554677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f121110-c6f8-4354-ae8b-64fc7d52791e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.561047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.562205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.562205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.562205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.562849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.564884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.626150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.628182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2478c-552f-4a36-9685-3055c20f51be service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.644480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 790/1624] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:02.649627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.650559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.651140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.679867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 835/1625] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:02.680737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.698021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109834cd-ab78-471f-bd3a-d9bf924791c2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.705295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.706093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.706425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.711923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.711923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.711923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.716306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:02.719416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:02.726581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 836/1626] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:02.731626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.741364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 791/1627] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:02.746776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3c922-8198-4652-8317-7b4ee4977e31 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.758506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.761107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.761630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.762070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] PATH_INFO: `/v3/projects/59838991323d4228be05d3b399b54224/users/991d1cd8279d4862899a870f43a8f0e6/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.768361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.780057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59838991323d4228be05d3b399b54224, user_id=991d1cd8279d4862899a870f43a8f0e6, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.784340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.839699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337897-33a7-4df5-8c2d-c0f6378bad80 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.845017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.845700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.845975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.846301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.860335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.861299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 837/1628] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:02 2022] PUT /identity/v3/projects/59838991323d4228be05d3b399b54224/users/991d1cd8279d4862899a870f43a8f0e6/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:02.861956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:02.865619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:02.877968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 792/1629] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:02.975150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f881d-f2f1-4678-b86a-ec75f5a40232 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:02.983471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:02.984204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:02.984624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:02.984961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:02.986432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:02.987696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c302b7-8a70-4c41-aa57-746380de0d6d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:03.006210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 838/1630] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:03.008610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:03.025645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c387-c3dd-4263-8863-d881cdebdfcc service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:03.036954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:03.041446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.041446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.041446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] PATH_INFO: `/v3/projects/59838991323d4228be05d3b399b54224/users/991d1cd8279d4862899a870f43a8f0e6/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.042531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=59838991323d4228be05d3b399b54224, user_id=991d1cd8279d4862899a870f43a8f0e6, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:03.048297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf18b0-8d74-4a07-8b90-ed236ce388ab admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:03.062863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 793/1631] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:03 2022] PUT /identity/v3/projects/59838991323d4228be05d3b399b54224/users/991d1cd8279d4862899a870f43a8f0e6/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:03.077411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.077411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.077411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.114655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:03.121686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706585e-be25-48c5-aeea-a7c79556cc00 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.122661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706585e-be25-48c5-aeea-a7c79556cc00 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.123047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706585e-be25-48c5-aeea-a7c79556cc00 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.124800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 794/1632] 10.222.0.75 () {54 vars in 912 bytes} [Thu Aug 25 03:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:06:03.130827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] MFA Rules not processed for user `991d1cd8279d4862899a870f43a8f0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:03.132446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:03.133231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.133631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.134137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.170375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:03.173329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:03.181444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:03.281550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 839/1633] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:03.298130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:03.313063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 795/1634] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:06:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:03.379223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee387-dacd-46cd-801e-4232eb71b6f7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:03.383613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:03.384630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.384974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.385353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.395995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:03.397684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:03.408790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 840/1635] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:03.602074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:03.620868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cde70-caf2-4705-9e15-557a827e9ef0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:03.628550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:03.629294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.631277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.631665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:03.657263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:03.660764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:03.682625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 796/1636] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:03.975951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:03.985961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe62-4fcd-426f-a356-aa37afc22895 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:03.992575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:03.993291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:03.993574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:03.993903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:04.121935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:04.121935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fe2c5-2b87-40d9-b1b2-bdcc762c7a0c service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:04.133144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 841/1637] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:04.609949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:04.609949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84e66b-3bef-441f-90a4-dc51a3e49bf4 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:04.609949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:04.609949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] PATH_INFO: `/v3/users/94a5fef1b4394d4b81d5ac52e94b6bf2` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a5fef1b4394d4b81d5ac52e94b6bf2)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: WARNING py.warnings [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: warnings.warn(msg) Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: Aug 25 03:06:04.621240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:04.866000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a5fef1b4394d4b81d5ac52e94b6bf2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:04.872667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a5fef1b4394d4b81d5ac52e94b6bf2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:04.889543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a5fef1b4394d4b81d5ac52e94b6bf2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:04.997451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a5fef1b4394d4b81d5ac52e94b6bf2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:04.998246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a5fef1b4394d4b81d5ac52e94b6bf2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:05.043417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 797/1638] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:04 2022] DELETE /identity/v3/users/94a5fef1b4394d4b81d5ac52e94b6bf2 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:05.048962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9b2d8-fc1c-4297-bbba-bb33f297f216 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:05.049422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9b2d8-fc1c-4297-bbba-bb33f297f216 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:05.049512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9b2d8-fc1c-4297-bbba-bb33f297f216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:05.078647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-77e9b2d8-fc1c-4297-bbba-bb33f297f216 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:05.080211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-77e9b2d8-fc1c-4297-bbba-bb33f297f216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:05.133038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:05.180199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 842/1639] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:05.200380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:05.202222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:05.224243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:05.276962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:05.277246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:05.277527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:05.317072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad4b9f-634d-470e-b9c5-71172df07c05 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:05.318276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] MFA Rules not processed for user `72e948509c3444eaacf205434e6db4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:05.319961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:05.326343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:05.328816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:05.329325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:05.330100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:05.359989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:05.362420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:05.392619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 798/1640] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:05.555990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 843/1641] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:05.561064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:05.562638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:05.590100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ef3c2-73bf-4772-9438-d98062e9be45 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:05.602462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:05.604461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:05.604955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:05.605448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:05.609063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:05.612382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:05.732776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 799/1642] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:05 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:05.734395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:05.758112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496a2cb-a228-4aa9-801e-511ead5cce7c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:05.879928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:05.880639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:05.880922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:05.881196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:05.881886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:05.886011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.075906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 844/1643] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:05 2022] POST /identity/v3/users => generated 309 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:06.079102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:06.109538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d94e9f-16c6-47aa-8153-5e18403b9ae0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:06.123761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:06.125233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.125712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.126038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.126903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:06.129130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.147773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 800/1644] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:06.152599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:06.180254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0144ff-1df7-45be-ab19-5a5ccce847d2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:06.189649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:06.192050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.192581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.193721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] PATH_INFO: `/v3/projects/3427539430d9466f87f943bfa38fbf87/users/bd444a74bfbe4abb856abd1ee5fd19cf/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.218829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427539430d9466f87f943bfa38fbf87, user_id=bd444a74bfbe4abb856abd1ee5fd19cf, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:06.222262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.281292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:06.286698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:06.344509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 845/1645] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:06 2022] PUT /identity/v3/projects/3427539430d9466f87f943bfa38fbf87/users/bd444a74bfbe4abb856abd1ee5fd19cf/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:06.348385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:06.354916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b8322-5200-43b2-af41-5a6debb39a1f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:06.445350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:06.463560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:06.464316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.464599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.465615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] PATH_INFO: `/v3/projects/44ad45bb838d4a27a8c085b28b5a227d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.469651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ad45bb838d4a27a8c085b28b5a227d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:06.479874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.519607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d9f7-979b-4b4a-afde-8d76c54fc5ed admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:06.527120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:06.527968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.529187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.529187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.529887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:06.532632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.556905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 846/1646] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:06.561059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:06.585887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a918-1b14-4f1f-a1aa-f050aa6503ed admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:06.602621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:06.604302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.604756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.605170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] PATH_INFO: `/v3/projects/3427539430d9466f87f943bfa38fbf87/users/bd444a74bfbe4abb856abd1ee5fd19cf/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.620860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427539430d9466f87f943bfa38fbf87, user_id=bd444a74bfbe4abb856abd1ee5fd19cf, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:06.631738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099040c-5951-42b4-b6fb-fc0c7acc2d01 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:06.648511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 847/1647] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:06 2022] PUT /identity/v3/projects/3427539430d9466f87f943bfa38fbf87/users/bd444a74bfbe4abb856abd1ee5fd19cf/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:06.725775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ad45bb838d4a27a8c085b28b5a227d', 'request_id': 'req-db6ee8f8-5413-4230-b60f-b27b9a937647'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:06.737822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ad45bb838d4a27a8c085b28b5a227d', 'request_id': 'req-db6ee8f8-5413-4230-b60f-b27b9a937647'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:06.856441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ad45bb838d4a27a8c085b28b5a227d', 'request_id': 'req-db6ee8f8-5413-4230-b60f-b27b9a937647'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:06.928012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] The token cache is being invalidate because project 44ad45bb838d4a27a8c085b28b5a227d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:06.928378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:06.932073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-db6ee8f8-5413-4230-b60f-b27b9a937647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:06.936933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 801/1648] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:06 2022] DELETE /identity/v3/projects/44ad45bb838d4a27a8c085b28b5a227d => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:06.939324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:06.939785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:06.940222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:06.977167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:06.979245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:07.133141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 848/1649] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:07.137157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.137450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.138246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.184949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:07.188609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:07.220505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 802/1650] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:07.225412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.238862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e13ea-cb52-40f9-ae33-8df086c4030a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.251820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.252919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.253380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.253845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.255393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:07.255882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.259643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.292076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 849/1651] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:07.296871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.310491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d67e2-7491-4d6d-86d2-771dd7d0a435 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.333484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.334404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.334404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.334768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.344738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.346730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.398982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.402045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:07.429586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4b06-e018-4212-92bb-4002229cdbdf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.440917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.441937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.442221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.442497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] PATH_INFO: `/v3/users/4b694ae9029848f683d475646e7cd589` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.448725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b694ae9029848f683d475646e7cd589)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.450989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.637341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 803/1652] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:07 2022] POST /identity/v3/projects => generated 365 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:07.638776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.736929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28840fc1-969d-407d-9147-9501cfdc78bf admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.742795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.745040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.745478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.747732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.748681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.751141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.852266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b694ae9029848f683d475646e7cd589'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:07.854468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b694ae9029848f683d475646e7cd589'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:07.873868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 804/1653] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:07 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:07.875370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.880893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b694ae9029848f683d475646e7cd589'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:07.895626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d5d5-aec7-4277-8076-0a9ec142552c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.904690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.905706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.906249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.906727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.907897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.909895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.936928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 805/1654] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:07.938345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.952276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ea326-7b07-40d7-b5fc-8c4eff9da9dd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:07.957913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:07.958977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:07.960095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:07.961195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:07.962113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:07.966113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:07.982243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 806/1655] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:07.983496 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b694ae9029848f683d475646e7cd589'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:07.984037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:07.984245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-17ac32bf-d359-4de6-ae8a-a3c31e4dd078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b694ae9029848f683d475646e7cd589'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:07.987145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:08.004837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc71e-25a4-4ac9-a59e-1ea949924ba1 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:08.013461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:08.014809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.015316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.016708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] PATH_INFO: `/v3/projects/16172cd75a4a468987813abc83766f68/users/01fa40e3f5bb488f89c356d0055c7249/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.018510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 850/1656] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:07 2022] DELETE /identity/v3/users/4b694ae9029848f683d475646e7cd589 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:08.022966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.022966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.023433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.051562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=16172cd75a4a468987813abc83766f68, user_id=01fa40e3f5bb488f89c356d0055c7249, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:08.057726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a9abd-316c-4fd7-badd-edd6e0bcc9af admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:08.060455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:08.062069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:08.148766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 851/1657] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:08.154152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 807/1658] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:07 2022] PUT /identity/v3/projects/16172cd75a4a468987813abc83766f68/users/01fa40e3f5bb488f89c356d0055c7249/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:08.157571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:08.159453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:08.169187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047483-64c8-4678-989f-628b225a2622 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.169482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047483-64c8-4678-989f-628b225a2622 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.169853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047483-64c8-4678-989f-628b225a2622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.210779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-61047483-64c8-4678-989f-628b225a2622 None None] MFA Rules not processed for user `01fa40e3f5bb488f89c356d0055c7249`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:08.240543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-61047483-64c8-4678-989f-628b225a2622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:08.266245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:08.362094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 808/1659] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:08.363467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61047483-64c8-4678-989f-628b225a2622 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:08.367021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b0c7e-c6b0-4a14-85cf-95eb1dd4d963 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:08.534666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:08.538348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.539439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.540683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.546170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61047483-64c8-4678-989f-628b225a2622 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:08.560886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:08.562517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.563377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.563563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.564474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:08.565651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:08.567289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:08.568603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:08.606882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 852/1660] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:08.763353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 809/1661] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:08 2022] POST /identity/v3/projects => generated 365 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:08.769917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:08.785610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075579d3-6936-442a-9338-835c6bcb732c service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:08.889558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:08.935699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:08.936401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:08.936682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:08.936997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:08.938511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:08.945288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:09.121107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:09.121537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 853/1662] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:08 2022] POST /identity/v3/users => generated 289 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:09.131807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64e38c-7e9c-4942-ae94-0e1d872aba56 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:09.148056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:09.148730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:09.149013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:09.149336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:09.150238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:09.154440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:09.177605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 810/1663] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:09.182419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:09.196958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739ca1-84bf-46ea-bc91-f7e6bc38e72c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:09.212998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:09.213945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:09.214379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:09.214819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] PATH_INFO: `/v3/projects/2e33017ca0484bb4b6ed0839cac8a048/users/b94778d2a8e349daa58521fd97fc4270/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:09.234097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e33017ca0484bb4b6ed0839cac8a048, user_id=b94778d2a8e349daa58521fd97fc4270, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:09.237121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:09.401323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 854/1664] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:09 2022] PUT /identity/v3/projects/2e33017ca0484bb4b6ed0839cac8a048/users/b94778d2a8e349daa58521fd97fc4270/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:09.405695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:09.424189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9639-b5a1-451b-bb66-035b0cb04c4b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:09.609262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:09.615622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:09.615622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:09.615622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:09.615622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:09.617946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25b853-3cc0-440c-8e44-268faf4a1c2a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:09.651798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:09.652257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 811/1665] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:09.669152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca294e6a-29a4-4aad-b6e7-04c8e796b034 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:09.683967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:09.686020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:09.692656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:09.692656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] PATH_INFO: `/v3/projects/2e33017ca0484bb4b6ed0839cac8a048/users/b94778d2a8e349daa58521fd97fc4270/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:09.692656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e33017ca0484bb4b6ed0839cac8a048, user_id=b94778d2a8e349daa58521fd97fc4270, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:09.699818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:09.737238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c727d-4f90-43e9-b492-e2a6918e450e None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:09.737530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c727d-4f90-43e9-b492-e2a6918e450e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:09.737805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c727d-4f90-43e9-b492-e2a6918e450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:09.741616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 855/1666] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:09 2022] PUT /identity/v3/projects/2e33017ca0484bb4b6ed0839cac8a048/users/b94778d2a8e349daa58521fd97fc4270/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:09.893372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4b0c727d-4f90-43e9-b492-e2a6918e450e None None] MFA Rules not processed for user `b94778d2a8e349daa58521fd97fc4270`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:09.909371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4b0c727d-4f90-43e9-b492-e2a6918e450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:10.248668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 812/1667] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:10.258144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:10.263964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:10.391812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fd5aa-2935-48ef-bbc5-259d57d9d834 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:10.401122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:10.402016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:10.402487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:10.402906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:10.432291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:10.436495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:10.468683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 856/1668] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:11.424328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645658-f57a-44d3-8118-f45725bcf508 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:11.425211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645658-f57a-44d3-8118-f45725bcf508 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:11.425723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18645658-f57a-44d3-8118-f45725bcf508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:11.497129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-18645658-f57a-44d3-8118-f45725bcf508 None None] MFA Rules not processed for user `bd444a74bfbe4abb856abd1ee5fd19cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:11.511465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-18645658-f57a-44d3-8118-f45725bcf508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:11.744913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 813/1669] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:11.891611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:12.021069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:12.022587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:12.161368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faaafc-ca2a-4367-962e-e50505210cc3 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:12.165617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:12.166408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:12.167992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:12.168639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] PATH_INFO: `/v3/projects/af2fb18a62fc4528955820da1bb6a409` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:12.170137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2fb18a62fc4528955820da1bb6a409)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:12.171252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: WARNING py.warnings [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 25 03:06:12.171252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: warnings.warn(msg) Aug 25 03:06:12.171252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: Aug 25 03:06:12.173660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:12.340621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2fb18a62fc4528955820da1bb6a409', 'request_id': 'req-ace04c75-8ca8-4024-a807-f5ec2fada248'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:12.352112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2fb18a62fc4528955820da1bb6a409', 'request_id': 'req-ace04c75-8ca8-4024-a807-f5ec2fada248'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:12.374909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:12.380943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2fb18a62fc4528955820da1bb6a409', 'request_id': 'req-ace04c75-8ca8-4024-a807-f5ec2fada248'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:12.497567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] The token cache is being invalidate because project af2fb18a62fc4528955820da1bb6a409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:12.498164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:12.499812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ace04c75-8ca8-4024-a807-f5ec2fada248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:12.502910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 857/1670] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:12 2022] DELETE /identity/v3/projects/af2fb18a62fc4528955820da1bb6a409 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:12.507652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:12.508065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:12.508337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:12.528125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:12.528887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:12.528887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:12.568350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:12.572748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:12.578136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] MFA Rules not processed for user `fe93db6503a143ab9ed50fedbc8f010f`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:12.583169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:12.757157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 858/1671] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:12.758003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:12.761897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:12.783913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 814/1672] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:12.785153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:12.787424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:12.962537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6a51a-7db6-4ca9-be5d-76b35845990b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:12.962537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:12.962537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:12.962537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:12.964264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:12.964511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bcb31-fd2c-4be6-bfd5-27adf23ab96a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:12.967094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:12.967760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:12.967882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:12.968056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:12.968364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] PATH_INFO: `/v3/users/01fa40e3f5bb488f89c356d0055c7249` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:12.969559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fa40e3f5bb488f89c356d0055c7249)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:12.969999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205dffe-cbfb-471a-bdf4-24f161a7d89e service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:12.971508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:12.986519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 815/1673] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:13.001148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.003636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.003929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.050142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:13.051478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:13.066684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 816/1674] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:13.068386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.088703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cbde-7abf-4d69-ba3c-aba0f368500f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.096579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.097211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.097490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.097774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.098504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:13.098815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.100865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:13.116621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 817/1675] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:13.122539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.139408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb162eb-d0a2-49e7-9d6e-3a819b823738 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.173689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.173689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.173689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.173689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.174227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.174227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:13.345818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 818/1676] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:13 2022] POST /identity/v3/projects => generated 367 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:13.348317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.391489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fa40e3f5bb488f89c356d0055c7249'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:13.393066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fa40e3f5bb488f89c356d0055c7249'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:13.420630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fa40e3f5bb488f89c356d0055c7249'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:13.512219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc1e0d-3d92-4571-9985-b0361211df76 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.527814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.528748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.529195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.529699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.530550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.534370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:13.539788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fa40e3f5bb488f89c356d0055c7249'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:13.541239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b737c9e4-b63c-4891-81ad-1add1ba6313a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fa40e3f5bb488f89c356d0055c7249'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:13.579804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 859/1677] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:12 2022] DELETE /identity/v3/users/01fa40e3f5bb488f89c356d0055c7249 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:13.585736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.587150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.587738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.636278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:13.639567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:13.682128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 819/1678] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:13 2022] POST /identity/v3/users => generated 284 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:13.685959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.691501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:13.712787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63167a-411d-4a6d-bb5b-59c6f40763d8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.781936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 860/1679] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:13.825485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.826563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.827217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.827708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.828745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.830647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:13.854959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 820/1680] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:13.856594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.869675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7608d-32a2-424f-ae60-f399c52f3d70 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.875900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.878139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.878139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.878139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.878408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.880229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:13.889984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.898829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:13.898966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 861/1681] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:13.902732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:13.916097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9354e-70ea-4c73-b95c-84486e90fed2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:13.922097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:13.923338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:13.923772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:13.924124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] PATH_INFO: `/v3/projects/53db003a70ac49f8b7a732e2f9834895/users/a2c282d85b5147e3a7e39ceaba657004/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:13.950402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53db003a70ac49f8b7a732e2f9834895, user_id=a2c282d85b5147e3a7e39ceaba657004, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:13.953662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd62e-92c4-48f1-abab-3bd78430a2d5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:14.007132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf11f2-1282-46c0-8683-8d6f04ecc1f7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:14.012903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:14.015439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.015439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.015439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.026418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:14.028434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f51fe-ee0e-41ca-a1d1-cc833a74b27b service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:14.044901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 821/1682] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:14.046983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc001-97fc-4d87-a5f7-617c4165db11 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.054384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc001-97fc-4d87-a5f7-617c4165db11 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.068190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 862/1683] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:13 2022] PUT /identity/v3/projects/53db003a70ac49f8b7a732e2f9834895/users/a2c282d85b5147e3a7e39ceaba657004/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:14.070692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97ce1-5008-431a-b81f-7cf5b813c61c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.071114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97ce1-5008-431a-b81f-7cf5b813c61c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.071395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97ce1-5008-431a-b81f-7cf5b813c61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.072625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc001-97fc-4d87-a5f7-617c4165db11 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.076424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 822/1684] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:06:14 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:14.096025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.096430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.096811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.140232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:14.140930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7af97ce1-5008-431a-b81f-7cf5b813c61c None None] MFA Rules not processed for user `a2c282d85b5147e3a7e39ceaba657004`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:14.143880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:14.144555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7af97ce1-5008-431a-b81f-7cf5b813c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:14.173043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:14.318217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 823/1685] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:06:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:14.318894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 863/1686] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:14.329260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:14.331088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:14.417689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06f46e-55d3-40e8-a1cd-44346ffabc55 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:14.421822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:14.422335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.423039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.423039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.434776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:14.437033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e500e0-cf62-4c89-b26e-89d325853d3a service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:14.450140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 824/1687] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:14.470677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eac06-375c-4e35-b3bd-21add328e32c None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.472240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eac06-375c-4e35-b3bd-21add328e32c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.473393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eac06-375c-4e35-b3bd-21add328e32c None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.477469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 864/1688] 10.222.0.75 () {54 vars in 961 bytes} [Thu Aug 25 03:06:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:06:14.490290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.490290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.490290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.564455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:14.568953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:14.609613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 825/1689] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Aug 25 03:06:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:14.624822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795eac06-375c-4e35-b3bd-21add328e32c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:14.645036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:14.660465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795eac06-375c-4e35-b3bd-21add328e32c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:14.675429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:14.676980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:14.677269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:14.677549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:14.703402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:14.705291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:14.732526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 865/1690] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:16.203076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:16.263769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:16.266413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:16.385044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:16.386454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:16.444733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338e937-94f2-4d01-b6ca-7c3611f12dfd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:16.451349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:16.452017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:16.452418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:16.452746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] PATH_INFO: `/v3/projects/16172cd75a4a468987813abc83766f68` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:16.453949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16172cd75a4a468987813abc83766f68)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:16.456522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:16.546791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eaf57-4460-4cd3-90c4-697614211cce service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:16.554485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:16.555335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:16.567511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:16.567511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] PATH_INFO: `/v3/users/72e948509c3444eaacf205434e6db4e4` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:16.567511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e948509c3444eaacf205434e6db4e4)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:16.567511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:16.688992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16172cd75a4a468987813abc83766f68', 'request_id': 'req-1507c2c7-e663-4c2c-8e21-870d37bbbc73'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.703090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16172cd75a4a468987813abc83766f68', 'request_id': 'req-1507c2c7-e663-4c2c-8e21-870d37bbbc73'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.888607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16172cd75a4a468987813abc83766f68', 'request_id': 'req-1507c2c7-e663-4c2c-8e21-870d37bbbc73'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.929639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] The token cache is being invalidate because project 16172cd75a4a468987813abc83766f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:16.936985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.943071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.949631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 826/1691] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:16 2022] DELETE /identity/v3/projects/16172cd75a4a468987813abc83766f68 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:16.950971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:16.954861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:16.977284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e948509c3444eaacf205434e6db4e4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:16.982645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e948509c3444eaacf205434e6db4e4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.020993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e948509c3444eaacf205434e6db4e4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.116984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e948509c3444eaacf205434e6db4e4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.117616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a0ccf820-4857-4b7c-9b76-641cfd4e4360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e948509c3444eaacf205434e6db4e4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.179819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507c2c7-e663-4c2c-8e21-870d37bbbc73 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:17.181561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 866/1692] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:16 2022] DELETE /identity/v3/users/72e948509c3444eaacf205434e6db4e4 => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:17.305502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:17.308446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:17.309014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:17.309495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] PATH_INFO: `/v3/users/b94778d2a8e349daa58521fd97fc4270` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:17.311094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94778d2a8e349daa58521fd97fc4270)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:17.319403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:17.666704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94778d2a8e349daa58521fd97fc4270'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.667396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94778d2a8e349daa58521fd97fc4270'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.689829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94778d2a8e349daa58521fd97fc4270'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.754863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:17.834141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94778d2a8e349daa58521fd97fc4270'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.836530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-5b698801-a0c4-4688-9f1f-e81adf8f17e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94778d2a8e349daa58521fd97fc4270'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:17.855800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:17.856303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:17.856516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:17.911518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 827/1693] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:16 2022] DELETE /identity/v3/users/b94778d2a8e349daa58521fd97fc4270 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:17.980137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] MFA Rules not processed for user `7daac509316048ec903a5d78d3586aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:17.990395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:18.203698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 867/1694] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:18.214701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:18.216881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:18.216881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:18.255956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] MFA Rules not processed for user `7daac509316048ec903a5d78d3586aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:18.257250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:18.280708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 828/1695] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:18.293203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:18.295070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:18.579411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0cb8a-c0ce-40b2-bad6-d438e7b7fddc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:18.602358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:18.605568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:18.605995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:18.607361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:18.637206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:18.641493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:18.683250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 868/1696] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:18.839728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:19.009635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:19.011719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:19.168802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c23b-6177-4bdb-b0ae-b7a59fbeea36 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:19.175539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:19.176457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:19.176745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:19.177024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] PATH_INFO: `/v3/projects/bc5b9a0d35974e5e83f4e739558d0a2d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:19.179782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5b9a0d35974e5e83f4e739558d0a2d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:19.190312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:19.296322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:19.301982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:19.320995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ed8c6-8db2-455e-b18a-02b7d6f2c84a service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:19.359771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:19.360772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:19.361056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:19.361382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] PATH_INFO: `/v3/projects/2e33017ca0484bb4b6ed0839cac8a048` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:19.364441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e33017ca0484bb4b6ed0839cac8a048)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:19.366583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:19.418225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5b9a0d35974e5e83f4e739558d0a2d', 'request_id': 'req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.441418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5b9a0d35974e5e83f4e739558d0a2d', 'request_id': 'req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.525895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e33017ca0484bb4b6ed0839cac8a048', 'request_id': 'req-05757657-8030-44d8-82d7-a52034fd46db'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.541095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e33017ca0484bb4b6ed0839cac8a048', 'request_id': 'req-05757657-8030-44d8-82d7-a52034fd46db'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.542456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e33017ca0484bb4b6ed0839cac8a048', 'request_id': 'req-05757657-8030-44d8-82d7-a52034fd46db'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.575428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5b9a0d35974e5e83f4e739558d0a2d', 'request_id': 'req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.586312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] The token cache is being invalidate because project bc5b9a0d35974e5e83f4e739558d0a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:19.587014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.588521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.590480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 829/1697] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:19 2022] DELETE /identity/v3/projects/bc5b9a0d35974e5e83f4e739558d0a2d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:19.597183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:19.601039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:19.673407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] The token cache is being invalidate because project 2e33017ca0484bb4b6ed0839cac8a048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:19.673800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.675325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-05757657-8030-44d8-82d7-a52034fd46db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:19.677380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 869/1698] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:19 2022] DELETE /identity/v3/projects/2e33017ca0484bb4b6ed0839cac8a048 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:19.678827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49259-e4e8-41c3-9b77-1e1cfd22cddd None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:19.679201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49259-e4e8-41c3-9b77-1e1cfd22cddd None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:19.679588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49259-e4e8-41c3-9b77-1e1cfd22cddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:19.765964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-01b49259-e4e8-41c3-9b77-1e1cfd22cddd None None] MFA Rules not processed for user `4529d06c4eda48f788ce7c0327366377`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:19.770072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-01b49259-e4e8-41c3-9b77-1e1cfd22cddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:19.782455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2b24d-b7a3-4d99-a1f3-7d083922bb39 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:19.896551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:19.897315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:19.897735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:19.898061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] PATH_INFO: `/v3/users/bd444a74bfbe4abb856abd1ee5fd19cf` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:19.899778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd444a74bfbe4abb856abd1ee5fd19cf)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:19.902234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:20.016808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 870/1699] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:20.022296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4f94-93de-4e7c-bc68-ba8b5694491d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.022672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4f94-93de-4e7c-bc68-ba8b5694491d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.022954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4f94-93de-4e7c-bc68-ba8b5694491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.074581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c70a4f94-93de-4e7c-bc68-ba8b5694491d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:20.076613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c70a4f94-93de-4e7c-bc68-ba8b5694491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.095656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 871/1700] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:20.097525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765f5d-5e17-4f5c-874d-66d9776265f5 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.097833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765f5d-5e17-4f5c-874d-66d9776265f5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.098114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8765f5d-5e17-4f5c-874d-66d9776265f5 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.100316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 872/1701] 10.222.0.75 () {54 vars in 912 bytes} [Thu Aug 25 03:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:20.101620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f403f-6969-43cb-98f5-130ebd40e5f8 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.101896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f403f-6969-43cb-98f5-130ebd40e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.102185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f403f-6969-43cb-98f5-130ebd40e5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.136898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1f0f403f-6969-43cb-98f5-130ebd40e5f8 None None] MFA Rules not processed for user `4529d06c4eda48f788ce7c0327366377`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:20.138655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1f0f403f-6969-43cb-98f5-130ebd40e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.147341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 873/1702] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:20.148923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b00fb-e184-45cf-b4f0-0555adb3ebe4 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.149215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b00fb-e184-45cf-b4f0-0555adb3ebe4 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.149638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b00fb-e184-45cf-b4f0-0555adb3ebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.199177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1d1b00fb-e184-45cf-b4f0-0555adb3ebe4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:20.200470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1d1b00fb-e184-45cf-b4f0-0555adb3ebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.226539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 874/1703] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:20.229969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.230342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.230692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.242574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd444a74bfbe4abb856abd1ee5fd19cf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:20.246490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd444a74bfbe4abb856abd1ee5fd19cf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:20.263337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd444a74bfbe4abb856abd1ee5fd19cf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:20.307277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:20.310429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.399446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd444a74bfbe4abb856abd1ee5fd19cf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:20.401001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd444a74bfbe4abb856abd1ee5fd19cf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:20.414416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 875/1704] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:06:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:20.416016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:20.417780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.436478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 830/1705] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:19 2022] DELETE /identity/v3/users/bd444a74bfbe4abb856abd1ee5fd19cf => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:20.438497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:20.441428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.523046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ae71-c3eb-4c92-9ddd-4322c0b8cedf None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:20.546264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:20.605808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37a247-2785-459c-9e7c-04a2b95546a5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:20.621928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:20.623050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.624455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.626402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.627293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:20.628287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:20.632920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:20.660336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 831/1706] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:20.661328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:20.676502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.764539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:20.764539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.769911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.770252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.776989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.821759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d546a-7222-4913-bacb-6dc57819b37e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:20.826723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:20.827547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.827915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.828211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.842634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:20.844113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:20.854676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 832/1707] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:20.855563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:20.876140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a8b39-627d-4693-9e81-bc93d0b61a86 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:20.885465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:20.886323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:20.886769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:20.887204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:20.888004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:20.902466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:20.957356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:20.958967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:20.969766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 876/1708] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:20.970681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:20.972028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:20.992079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76d2a-a2c3-47f6-b679-380d00caa9a3 service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:20.999295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:21.000132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.000829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.001124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] PATH_INFO: `/v3/projects/3427539430d9466f87f943bfa38fbf87` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.002499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3427539430d9466f87f943bfa38fbf87)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:21.004942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:21.037659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 833/1709] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:20 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:21.042066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:21.129253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714492-82b0-4164-ac14-7a70778fec67 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:21.135291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:21.136008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.136504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.136850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.137625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:21.139751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:21.176114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3427539430d9466f87f943bfa38fbf87', 'request_id': 'req-a57a9245-d392-4f98-8dcc-95138d2eb432'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:21.182416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3427539430d9466f87f943bfa38fbf87', 'request_id': 'req-a57a9245-d392-4f98-8dcc-95138d2eb432'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:21.191310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3427539430d9466f87f943bfa38fbf87', 'request_id': 'req-a57a9245-d392-4f98-8dcc-95138d2eb432'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:21.359884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] The token cache is being invalidate because project 3427539430d9466f87f943bfa38fbf87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:21.361098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:21.361596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 834/1710] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:21 2022] POST /identity/v3/users => generated 285 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:21.362627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a57a9245-d392-4f98-8dcc-95138d2eb432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:21.367698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:21.369023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 877/1711] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:20 2022] DELETE /identity/v3/projects/3427539430d9466f87f943bfa38fbf87 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:21.371138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.371649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.372092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.379576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:21.407479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2b669-dea2-48ef-aed2-a783139268fd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:21.480873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:21.486814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:21.644313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:21.645380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.645863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.646916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.654779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:21.658086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88633d-4c3a-4d7a-a26e-0a49b6c3c98d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:21.688938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 835/1712] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:21.693116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.693431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.693715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.703242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 878/1713] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:21.706350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:21.737529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214526e6-9837-465f-bbbe-d9950a24159f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:21.749976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:21.750630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.751070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.751368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.752061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:21.753756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616fac4-b6e6-449f-9480-b9d76f61d8d9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:21.769362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] MFA Rules not processed for user `804df9333a474723968f97eb9d6bfc69`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:21.770898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:21.775993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 879/1714] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:21.777858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.777967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.778188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.813927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:21.815203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:21.840399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 880/1715] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:21.841048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:21.864632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640179d9-d6c4-43de-a250-d74735295a5e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:21.876836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:21.878594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:21.878594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:21.878594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] PATH_INFO: `/v3/projects/9e334afb613b42f7b9193cb0161cea2a/users/20148474884645d784eb6ac3e1fd8299/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:21.920533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e334afb613b42f7b9193cb0161cea2a, user_id=20148474884645d784eb6ac3e1fd8299, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:21.923266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d65db-b8ac-4da8-9c53-a7ad18164a69 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:21.972693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 836/1716] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:21.973712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:21.995839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb2b6-f169-47d1-9bd4-8e72792890b3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.018604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 881/1717] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:21 2022] PUT /identity/v3/projects/9e334afb613b42f7b9193cb0161cea2a/users/20148474884645d784eb6ac3e1fd8299/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:22.021334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb681d-969e-4b1d-b7ed-98ad8effdf44 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.021334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb681d-969e-4b1d-b7ed-98ad8effdf44 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.021680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb681d-969e-4b1d-b7ed-98ad8effdf44 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.023538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 882/1718] 10.222.0.75 () {54 vars in 912 bytes} [Thu Aug 25 03:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:22.031308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.032431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53cbc-fc1a-4a13-b8ec-6798fefe982a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.032728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53cbc-fc1a-4a13-b8ec-6798fefe982a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.033027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa53cbc-fc1a-4a13-b8ec-6798fefe982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.034501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.034907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.035279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.036312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:22.036723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:22.038479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b146d4a-c55b-47d1-b333-2e25ae985194 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:22.057359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 837/1719] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:22.058982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.059375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.059862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.116294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:22.122224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.122360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-daa53cbc-fc1a-4a13-b8ec-6798fefe982a None None] MFA Rules not processed for user `20148474884645d784eb6ac3e1fd8299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:22.122360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-daa53cbc-fc1a-4a13-b8ec-6798fefe982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.302686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 838/1720] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:06:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:22.303562 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:22.355739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 883/1721] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:22.357628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.357920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.364322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.380462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] MFA Rules not processed for user `fe93db6503a143ab9ed50fedbc8f010f`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:22.381898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.397232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f5df-52a0-4d4c-ac1f-b34fa0f08171 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.407808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.408678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.409317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.409651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.410312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:22.413153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:22.473712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 884/1722] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:22.474597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:22.484808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.584059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 839/1723] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:22 2022] POST /identity/v3/projects => generated 393 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:22.585006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:22.591349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.611875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6694-0b37-4104-bee8-e6722ef12fa9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.721162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6e9c-ff1c-414a-8c4f-b9a0370c7de2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.726310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.727757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.728657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.729170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.740438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:22.742767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:22.755149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.756493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.756999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.757457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.763688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.765440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:22.768672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 840/1724] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:22.778543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:22.778543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:22.783082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:22.784970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:22.806730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 885/1725] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:22.807620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:22.940067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e46-b2c6-4e2e-8710-186ef970497b service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.953918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.956137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.956936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.957357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.970276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d6a16-8a5e-485d-942d-25ed1cef18d5 service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:22.984429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:22.985350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:22.985680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:22.985947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:22.986587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:22.992793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.014946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:23.101791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.104086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.125232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 841/1726] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:23.126200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.147197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64146174-8e63-4859-ac8f-3fb9e48bea99 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.162095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.162728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.163115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.163545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.166050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.168062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.206581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 886/1727] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:22 2022] POST /identity/v3/projects => generated 367 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:23.210665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.312999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16cba2-190b-4e0a-affc-7369db8082ad admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.319222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.320483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.320948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.321406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.322386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.324578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.422819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 842/1728] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:23 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:23.425344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.450815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa80e70-d88e-44bb-a509-213efafe97b2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.463439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.464387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.464674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.464957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.465674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.467480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.486130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 843/1729] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:23.490710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.512949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054da518-be47-4b90-9dac-15ebf2ad3fc8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.520985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.523009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.523398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.525651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.528311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.535800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69978c59-389a-4e0a-be1a-300a08f83e89 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.539122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 887/1730] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:23 2022] POST /identity/v3/users => generated 290 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:23.543020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.567850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ecd9-3fd0-48bc-aade-7d4477281a7e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.583945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.587549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.587703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 844/1731] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:23.587795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.587795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.587953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.587953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.587953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.590673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.596183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.632967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 888/1732] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:23.635071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.646661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] MFA Rules not processed for user `7daac509316048ec903a5d78d3586aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:23.647904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:23.656993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1239c7b-2e43-4a97-a44f-07f716e7e61a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.688033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.688751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.689366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.689780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] PATH_INFO: `/v3/projects/dd158f010f474763bf31af3e7bd48ab4/users/07dee131eaa8487c9de22cc00c5e5a36/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.763733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd158f010f474763bf31af3e7bd48ab4, user_id=07dee131eaa8487c9de22cc00c5e5a36, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.767032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69a686-d3f4-4647-9ed4-4ac75ff0f4d1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:23.831532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 845/1733] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:23.832414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:23.853883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a295-6411-45fa-8b55-2518d095d2e2 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:23.865448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:23.868924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.869246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.869513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] PATH_INFO: `/v3/projects/a5169cf8d47f4f5fbf934effb4b76093/users/ed622a7d358044789b1e0860b1b2dc5b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.923768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 889/1734] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:23 2022] PUT /identity/v3/projects/dd158f010f474763bf31af3e7bd48ab4/users/07dee131eaa8487c9de22cc00c5e5a36/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:23.934598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5169cf8d47f4f5fbf934effb4b76093, user_id=ed622a7d358044789b1e0860b1b2dc5b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:23.938743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:23.939033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:23.939307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:23.966015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:24.037317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] MFA Rules not processed for user `07dee131eaa8487c9de22cc00c5e5a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:24.038591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:24.119935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 846/1735] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:23 2022] PUT /identity/v3/projects/a5169cf8d47f4f5fbf934effb4b76093/users/ed622a7d358044789b1e0860b1b2dc5b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:24.122865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:24.306711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 890/1736] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:24.308856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:24.373127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ab6e-b4bc-442e-890d-ce5fbf9b4b8f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:24.379851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:24.380954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:24.381434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:24.381994 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:24.382795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:24.386238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:24.416448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 847/1737] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:24.419058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:24.444415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29cb56-63e1-4322-b876-e144547e9887 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:24.463593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:24.465062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:24.465360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:24.465645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] PATH_INFO: `/v3/projects/a5169cf8d47f4f5fbf934effb4b76093/users/ed622a7d358044789b1e0860b1b2dc5b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:24.474630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5169cf8d47f4f5fbf934effb4b76093, user_id=ed622a7d358044789b1e0860b1b2dc5b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:24.479363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cbe25-61dd-4cef-ba7d-4b438a9b62b5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:24.499029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 848/1738] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:24 2022] PUT /identity/v3/projects/a5169cf8d47f4f5fbf934effb4b76093/users/ed622a7d358044789b1e0860b1b2dc5b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:24.510585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:24.511134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:24.511573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:24.513427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd09898-a75e-4b62-aaea-b596afca4265 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:24.624454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] MFA Rules not processed for user `ed622a7d358044789b1e0860b1b2dc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:24.625784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:24.688183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:24.689243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:24.689243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:24.689426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:24.689984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:24.696552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:24.787829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 849/1739] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:24.799951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:24.802432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:24.944653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd7298-4fd0-4142-8a51-fea7e479b8f4 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:24.944766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 891/1740] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:24 2022] POST /identity/v3/projects => generated 393 bytes in 633 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:24.945916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:25.056097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:25.056642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:25.056964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:25.057243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:25.090201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e685-e518-4791-b9a1-a60374a83af0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:25.140554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:25.145615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:25.149948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:25.150735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:25.157737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:25.162915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:25.199557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:25.201961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:25.216516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 850/1741] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:25.319479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 892/1742] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:24 2022] POST /identity/v3/users => generated 303 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:25.323847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:25.348250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a347a-0f58-4a6a-b372-b63f0b436c01 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:25.363628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:25.365455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:25.367197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:25.367569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:25.370177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:25.374624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:25.396514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 851/1743] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:25.401747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:25.428760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b7d4-7fcc-4de1-a6f1-e26052d7cca6 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:25.444660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:25.445675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:25.446696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:25.447720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] PATH_INFO: `/v3/projects/f6988bf260544a81bbec9b1af8873824/users/156dff03dae54e3ba7ad64f73bb12ea9/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:25.485991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6988bf260544a81bbec9b1af8873824, user_id=156dff03dae54e3ba7ad64f73bb12ea9, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:25.489607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:25.502336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:25.609923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 893/1744] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:25 2022] PUT /identity/v3/projects/f6988bf260544a81bbec9b1af8873824/users/156dff03dae54e3ba7ad64f73bb12ea9/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:25.613696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:25.682916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a53d-d256-4940-9da9-e81ec53aa6f8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:25.988864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:25.989504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:25.989792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:25.990070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:25.990741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:25.999316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77338c3-ccdb-4f55-8316-ba453d20f523 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:26.016267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 852/1745] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:26.018110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:26.053412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f612ee3-5b2b-410f-b9e6-a5ebe8e794c1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:26.082643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:26.083345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:26.083629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:26.083908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] PATH_INFO: `/v3/projects/f6988bf260544a81bbec9b1af8873824/users/156dff03dae54e3ba7ad64f73bb12ea9/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:26.104503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6988bf260544a81bbec9b1af8873824, user_id=156dff03dae54e3ba7ad64f73bb12ea9, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:26.116859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:26.133340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 894/1746] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:26 2022] PUT /identity/v3/projects/f6988bf260544a81bbec9b1af8873824/users/156dff03dae54e3ba7ad64f73bb12ea9/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:26.144006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:26.144376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:26.144610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:26.180100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] MFA Rules not processed for user `156dff03dae54e3ba7ad64f73bb12ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:26.180100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:26.360248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 853/1747] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:26.514931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:26.664420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b16dc2-ecb7-48b2-8417-ec3faebe7537 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:26.717746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:26.718409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:26.718670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:26.718943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:26.768363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:26.773462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020debb-95e0-4174-8c08-b39a8dd6a3af service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:26.808849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 895/1748] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:27.049340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:27.052429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:27.072682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ab772-ba6f-4df2-ad80-0ce1cf51a52d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:27.077252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046012-8786-417c-9822-c508d807b129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:27.078236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046012-8786-417c-9822-c508d807b129 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:27.078801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046012-8786-417c-9822-c508d807b129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:27.079492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046012-8786-417c-9822-c508d807b129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:27.143168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:27.257106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046012-8786-417c-9822-c508d807b129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:27.259299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046012-8786-417c-9822-c508d807b129 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:27.290509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 854/1749] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:28.575305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:28.576193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:28.577255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:28.625412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] MFA Rules not processed for user `804df9333a474723968f97eb9d6bfc69`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:28.645215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:28.881053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 896/1750] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:28.975809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832c0bf-801b-41db-b979-b29118ca3b33 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:28.978001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832c0bf-801b-41db-b979-b29118ca3b33 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:28.979125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832c0bf-801b-41db-b979-b29118ca3b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:29.131157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1832c0bf-801b-41db-b979-b29118ca3b33 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:29.136453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1832c0bf-801b-41db-b979-b29118ca3b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:29.289285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 855/1751] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:29.852207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:29.853663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:29.996462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349c20f-7d7f-4c7c-ac3b-9e4d84c83849 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:30.002694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:30.003630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:30.004105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:30.005051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:30.018959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:30.022429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:30.042058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 897/1752] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:06:30.191793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:30.844762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:30.845244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:30.845334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:30.896059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:30.898606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:31.171187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 856/1753] 10.222.0.79 () {58 vars in 1015 bytes} [Thu Aug 25 03:06:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:31.220113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:31.225375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:31.247695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7730-fdea-4f75-8428-dadc59deb59c service swift] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:31.301734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:31.304994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:31.308645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:31.308969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:31.369045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:31.372063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a68a8e-a608-4809-88d4-7c38123c8818 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:31.403915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 898/1754] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:32.707935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:34.424383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:34.431856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:34.694298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb2146-86d4-4acb-8480-0088af8e67f8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:34.699901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:34.700581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:34.700866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:34.701144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] PATH_INFO: `/v3/users/a2c282d85b5147e3a7e39ceaba657004` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:34.709057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c282d85b5147e3a7e39ceaba657004)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:34.711915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:35.072008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:35.123528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c282d85b5147e3a7e39ceaba657004'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:35.128365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c282d85b5147e3a7e39ceaba657004'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:35.151599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c282d85b5147e3a7e39ceaba657004'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:35.307513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c282d85b5147e3a7e39ceaba657004'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:35.308090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c282d85b5147e3a7e39ceaba657004'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:35.363603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 857/1755] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:34 2022] DELETE /identity/v3/users/a2c282d85b5147e3a7e39ceaba657004 => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:35.369331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421700-73ee-41fc-a27a-f8c2ed4d3498 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:35.369662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421700-73ee-41fc-a27a-f8c2ed4d3498 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:35.370186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421700-73ee-41fc-a27a-f8c2ed4d3498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:35.421348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7c421700-73ee-41fc-a27a-f8c2ed4d3498 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:35.425102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7c421700-73ee-41fc-a27a-f8c2ed4d3498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:35.604127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 899/1756] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:35.628613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:35.628613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:35.801553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41e52e-2e0e-4d35-b05b-32c378ddf8f0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:35.815519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:35.818505 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:35.821371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:35.824289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:35.848838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:35.850445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd22f77-7c78-4a0b-8d20-3823bdad65d9 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:35.894539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 858/1757] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:36.420368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.420943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.420943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.423020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 900/1758] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:36.452695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e85f6-e616-4f42-87fd-56144343bc04 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.453018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e85f6-e616-4f42-87fd-56144343bc04 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.454190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e85f6-e616-4f42-87fd-56144343bc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.461315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:36.464786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:36.519111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e55e85f6-e616-4f42-87fd-56144343bc04 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:36.523474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e55e85f6-e616-4f42-87fd-56144343bc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:36.532729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8718566-10d3-4cb5-b572-3b99826323a7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:36.543851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:36.552987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.552987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.552987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] PATH_INFO: `/v3/users/20148474884645d784eb6ac3e1fd8299` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.552987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=20148474884645d784eb6ac3e1fd8299)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:36.552987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:36.649238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 859/1759] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:06:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:36.649238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14b1e-d985-4cd6-b257-21185f31ca34 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.649238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14b1e-d985-4cd6-b257-21185f31ca34 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.649238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a14b1e-d985-4cd6-b257-21185f31ca34 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.650736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 860/1760] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:36.659790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.660882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.661492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.701052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:36.702849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:36.720542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 861/1761] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:06:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:36.721662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:36.725053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:36.745073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339549-4972-4dca-be13-36cce2d2c571 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:36.754523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:36.755436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:36.755865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:36.756539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:36.769578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:36.771434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f00d4-869f-423e-9c08-ab2ddeb82735 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:36.790090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 862/1762] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:36.909834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20148474884645d784eb6ac3e1fd8299'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:36.911593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20148474884645d784eb6ac3e1fd8299'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:36.932140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20148474884645d784eb6ac3e1fd8299'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:37.120660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20148474884645d784eb6ac3e1fd8299'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:37.127708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20148474884645d784eb6ac3e1fd8299'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:37.182714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 901/1763] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:36 2022] DELETE /identity/v3/users/20148474884645d784eb6ac3e1fd8299 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:37.185923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:37.186282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:37.187285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:37.221031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:37.222623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:37.321329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 863/1764] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:37.348253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:37.350105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:37.361683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:37.363753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:37.470541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e70904-0ba3-4fb8-b84b-eca7adec6dea admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:37.482083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:37.483123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:37.483764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:37.484329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:37.504756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:37.509004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:37.565698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 902/1765] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:37.575436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524f19-020b-4105-a42e-546ea5d6e3e3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:37.583118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:37.583745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:37.584551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:37.584837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:37.588630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:37.606785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:37.608997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:37.677074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 864/1766] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:37.896437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:38.066308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:38.068653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:38.085361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bb27-be18-40b6-aa3c-d432b7e5babe service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:38.105387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:38.105387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:38.105387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:38.105387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] PATH_INFO: `/v3/users/07dee131eaa8487c9de22cc00c5e5a36` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:38.105843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07dee131eaa8487c9de22cc00c5e5a36)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:38.106129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:38.107158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:38.109513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:38.161508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76ffa-b3dc-4c19-acf5-16a4c8b33c95 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:38.177341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:38.178507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:38.179135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:38.181953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] PATH_INFO: `/v3/projects/53db003a70ac49f8b7a732e2f9834895` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:38.186670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53db003a70ac49f8b7a732e2f9834895)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:38.203510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:38.562856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07dee131eaa8487c9de22cc00c5e5a36'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.564903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07dee131eaa8487c9de22cc00c5e5a36'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.572527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53db003a70ac49f8b7a732e2f9834895', 'request_id': 'req-7f248512-1a51-48a0-b13e-d395096777a8'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.581228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07dee131eaa8487c9de22cc00c5e5a36'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.584774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53db003a70ac49f8b7a732e2f9834895', 'request_id': 'req-7f248512-1a51-48a0-b13e-d395096777a8'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.718940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07dee131eaa8487c9de22cc00c5e5a36'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.719628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0308c2c-5cff-4b85-a69d-b521432cab24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07dee131eaa8487c9de22cc00c5e5a36'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.762252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:38.774162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 903/1767] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:38 2022] DELETE /identity/v3/users/07dee131eaa8487c9de22cc00c5e5a36 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:38.776270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:38.776826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:38.777101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:38.810221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:38.816270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:38.855624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53db003a70ac49f8b7a732e2f9834895', 'request_id': 'req-7f248512-1a51-48a0-b13e-d395096777a8'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.871250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] The token cache is being invalidate because project 53db003a70ac49f8b7a732e2f9834895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:38.884270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.885807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7f248512-1a51-48a0-b13e-d395096777a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:38.900017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 865/1768] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:38 2022] DELETE /identity/v3/projects/53db003a70ac49f8b7a732e2f9834895 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:38.905460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c750b9-6164-47a0-9107-61b053c225ac None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:38.913700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c750b9-6164-47a0-9107-61b053c225ac None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:38.913700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c750b9-6164-47a0-9107-61b053c225ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:38.914047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 904/1769] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:38.943388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:38.952737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:38.952890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c6c750b9-6164-47a0-9107-61b053c225ac None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:38.962295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c6c750b9-6164-47a0-9107-61b053c225ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:38.980995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 866/1770] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:38.983499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:38.983861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:38.984248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.010613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:39.012768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:39.025795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 867/1771] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:39.034162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.041497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5862fa5-6057-4f53-b560-574b68dfccab None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.042479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49f9fe-aee7-490b-9286-5f36adc52672 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.046884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.047610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.047927 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.048322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.049041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:39.049355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.051397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.068577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.073599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.073599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.073599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.096333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 868/1772] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:06:39.098459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.102872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:39.125659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66152-9c78-444c-b3ea-589c58ae7c7a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.129619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.135060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.135681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.135965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.136270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] PATH_INFO: `/v3/projects/9e334afb613b42f7b9193cb0161cea2a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.137901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e334afb613b42f7b9193cb0161cea2a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.139778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.142748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.186647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 905/1773] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:39.189505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.226052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826f54-a7e5-48de-b422-d826ff47b641 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.239963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.243676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.251818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.251818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.251818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.251818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.361427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e334afb613b42f7b9193cb0161cea2a', 'request_id': 'req-b933ed98-bb1d-46d2-8b4d-9c37835cf765'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:39.372705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e334afb613b42f7b9193cb0161cea2a', 'request_id': 'req-b933ed98-bb1d-46d2-8b4d-9c37835cf765'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:39.477461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 906/1774] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:39 2022] POST /identity/v3/projects => generated 363 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:39.484413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.505238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e334afb613b42f7b9193cb0161cea2a', 'request_id': 'req-b933ed98-bb1d-46d2-8b4d-9c37835cf765'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:39.530551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] The token cache is being invalidate because project 9e334afb613b42f7b9193cb0161cea2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:39.530551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:39.534592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:39.536946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 869/1775] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:39 2022] DELETE /identity/v3/projects/9e334afb613b42f7b9193cb0161cea2a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:39.540523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.542130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:39.646403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7d94-d9c4-48c7-822d-e7d30bc74656 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.653516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.654526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.655051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.655565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.656452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.659254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.739204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933ed98-bb1d-46d2-8b4d-9c37835cf765 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.759713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.759713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.759713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.759713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] PATH_INFO: `/v3/users/ed622a7d358044789b1e0860b1b2dc5b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.761744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed622a7d358044789b1e0860b1b2dc5b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.761744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.832876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 907/1776] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:39 2022] POST /identity/v3/users => generated 282 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:39.834064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.836698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:39.856469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7443d4-3703-4e16-995a-9411ebcc6440 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.869107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.869788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.870058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.870326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.871013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.872641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.884356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 908/1777] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:39.912945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:39.956437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec02f52-6064-4c93-9e3b-4a225125b290 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:39.965043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:39.966102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:39.966444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:39.966723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:39.967585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:39.970467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:39.991333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 909/1778] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:39.995712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:40.010964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5dfce-d500-4e11-aed3-c2a91cf273fa admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:40.025222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:40.026118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:40.026554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:40.028009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] PATH_INFO: `/v3/projects/e759fac3354b4a488ab40788db8db563/users/7eb99be8312142a6adc144675472a1ab/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:40.060469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e759fac3354b4a488ab40788db8db563, user_id=7eb99be8312142a6adc144675472a1ab, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:40.072229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa02ee-5aaa-487f-9262-58f50379c19a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:40.233453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed622a7d358044789b1e0860b1b2dc5b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:40.235380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed622a7d358044789b1e0860b1b2dc5b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:40.246543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed622a7d358044789b1e0860b1b2dc5b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:40.270719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 910/1779] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:39 2022] PUT /identity/v3/projects/e759fac3354b4a488ab40788db8db563/users/7eb99be8312142a6adc144675472a1ab/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:40.273359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:40.274436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:40.274714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:40.339519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] MFA Rules not processed for user `7eb99be8312142a6adc144675472a1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:40.341882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:40.441955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed622a7d358044789b1e0860b1b2dc5b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:40.443180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed622a7d358044789b1e0860b1b2dc5b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:40.490500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 870/1780] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:39 2022] DELETE /identity/v3/users/ed622a7d358044789b1e0860b1b2dc5b => generated 0 bytes in 950 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:40.506671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 911/1781] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:40.512181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:40.514115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:40.679598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05720b85-4abc-4332-9b09-b2c41f4dee04 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:40.686642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:40.687387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:40.688231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:40.688231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:40.689006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:40.691248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:40.882347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 871/1782] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:40 2022] POST /identity/v3/projects => generated 363 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:40.887040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:41.038218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62fc6d-881e-424a-af23-eb3a0b5f5ef7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:41.203773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:41.205577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:41.205577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:41.206639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:41.208091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:41.214395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:41.948251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 912/1783] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:40 2022] POST /identity/v3/users => generated 288 bytes in 1063 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:41.948988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:41.951022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:41.951976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:41.969023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992f209-de8c-4f17-b05a-a70b012d3120 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:41.973033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6bd9-b0a2-4d48-affc-5a481d2bdf51 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:41.975206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:41.976353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:41.977296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:41.977722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] PATH_INFO: `/v3/projects/dd158f010f474763bf31af3e7bd48ab4` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:41.979141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd158f010f474763bf31af3e7bd48ab4)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:41.984812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:41.999495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.000849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.001170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.001898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.002581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.004224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.018689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 913/1784] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:42.019790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.030648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3af6-b3c7-49b4-a284-876244885ef7 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.035239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.035874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.036169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.036458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] PATH_INFO: `/v3/projects/7a774755645e46b3838237a5cd1f1fb6/users/f0d467533cc24f75ab5bafb0672a369a/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.051273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a774755645e46b3838237a5cd1f1fb6, user_id=f0d467533cc24f75ab5bafb0672a369a, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.053970 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.238518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 914/1785] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:42 2022] PUT /identity/v3/projects/7a774755645e46b3838237a5cd1f1fb6/users/f0d467533cc24f75ab5bafb0672a369a/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:42.243645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.254581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd158f010f474763bf31af3e7bd48ab4', 'request_id': 'req-dec9225d-1854-4855-8a23-3973cbdc9bff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.263686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd158f010f474763bf31af3e7bd48ab4', 'request_id': 'req-dec9225d-1854-4855-8a23-3973cbdc9bff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.369407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c54788-6e23-46d6-92f7-8b8595cbb6cf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.443182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.444767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.445141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.445654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.446148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.451754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.463517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 915/1786] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:42.464707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.478301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4899-5bb0-4444-869b-f0e0c68a5a3a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.478401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd158f010f474763bf31af3e7bd48ab4', 'request_id': 'req-dec9225d-1854-4855-8a23-3973cbdc9bff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.482552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.483941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.483941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.483941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] PATH_INFO: `/v3/projects/7a774755645e46b3838237a5cd1f1fb6/users/f0d467533cc24f75ab5bafb0672a369a/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.486129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a774755645e46b3838237a5cd1f1fb6, user_id=f0d467533cc24f75ab5bafb0672a369a, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.489029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350aaa0-60c2-40db-b01f-756b7eaebbe4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.505410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 916/1787] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:42 2022] PUT /identity/v3/projects/7a774755645e46b3838237a5cd1f1fb6/users/f0d467533cc24f75ab5bafb0672a369a/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:42.514182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] The token cache is being invalidate because project dd158f010f474763bf31af3e7bd48ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:42.515145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.517868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.518022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.518022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.518375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.525232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 872/1788] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:41 2022] DELETE /identity/v3/projects/dd158f010f474763bf31af3e7bd48ab4 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:42.529859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.534509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:42.551475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9225d-1854-4855-8a23-3973cbdc9bff admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.557729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] MFA Rules not processed for user `f0d467533cc24f75ab5bafb0672a369a`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:42.559306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:42.641965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.642978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.643465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.644197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] PATH_INFO: `/v3/users/156dff03dae54e3ba7ad64f73bb12ea9` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.645499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=156dff03dae54e3ba7ad64f73bb12ea9)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.649091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.666702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 917/1789] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:42.674611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.678462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:42.795702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aba3b-b17e-46a3-be9f-7755cb63b01f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.800371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.801121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.801398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.801901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.812532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.813700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.836266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 918/1790] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:42.839723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:42.841489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:42.860445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0795f0-d4f4-4fbc-8612-f5f33fbe1239 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:42.864518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:42.865208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:42.865502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:42.869296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] PATH_INFO: `/v3/projects/a5169cf8d47f4f5fbf934effb4b76093` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:42.869296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5169cf8d47f4f5fbf934effb4b76093)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:42.869577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:42.909496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156dff03dae54e3ba7ad64f73bb12ea9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.910171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156dff03dae54e3ba7ad64f73bb12ea9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.917926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156dff03dae54e3ba7ad64f73bb12ea9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.979285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5169cf8d47f4f5fbf934effb4b76093', 'request_id': 'req-3df965c9-dde0-4897-9dec-1a93a8bf72d7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.984829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5169cf8d47f4f5fbf934effb4b76093', 'request_id': 'req-3df965c9-dde0-4897-9dec-1a93a8bf72d7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:42.990973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5169cf8d47f4f5fbf934effb4b76093', 'request_id': 'req-3df965c9-dde0-4897-9dec-1a93a8bf72d7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:43.053494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156dff03dae54e3ba7ad64f73bb12ea9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:43.054542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156dff03dae54e3ba7ad64f73bb12ea9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:43.073458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 873/1791] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:42 2022] DELETE /identity/v3/users/156dff03dae54e3ba7ad64f73bb12ea9 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:43.108563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.110435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:43.172408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c37a3-b398-466c-97d3-357922be63ff admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.176068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.176732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.177019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.177320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.179143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:43.195400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] The token cache is being invalidate because project a5169cf8d47f4f5fbf934effb4b76093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:43.195924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:43.197966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3df965c9-dde0-4897-9dec-1a93a8bf72d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:43.202913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 919/1792] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:42 2022] DELETE /identity/v3/projects/a5169cf8d47f4f5fbf934effb4b76093 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:43.204687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.204977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.205259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.227651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:43.229526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:43.249557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.251342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c636f4-ff9c-4a6a-b5f7-71a91ad30988 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.264237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 874/1793] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:43.298090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 920/1794] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:43.306761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.307191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.307688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.331082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:43.332627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:43.341353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 875/1795] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:43.345811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.355536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96d3d-cc0e-4028-913c-aa718fd0d492 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.360014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.360641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.360918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.361187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.361824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:43.362124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.363688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.375465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 921/1796] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:43.379958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.390223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c1fbf-39a7-4e82-9997-3a1cb9d29d45 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.395016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.395648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.395932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.396262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.396915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.398829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.561167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 876/1797] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:43 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:43.562634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.572854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bbd98-b044-471f-a2b7-724bed3e19df admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.644278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.644903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.645185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.645459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.646099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.648274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.718331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.720315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:43.733496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f00c3-e7ca-402c-825c-0298bb36c9ae admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.738468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.739040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.739352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.739583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] PATH_INFO: `/v3/projects/f6988bf260544a81bbec9b1af8873824` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.740778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6988bf260544a81bbec9b1af8873824)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.742695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.758076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 922/1798] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:43 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:43.765785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.778443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c663f-8967-411d-a717-1dddc9626dc9 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.783269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.783883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.784222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.784499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.785216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.786813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.800186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 923/1799] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:43.801740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.812377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c814b-b02a-4e45-974b-bc98a7e18b4d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.817406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.818017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.818298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.818575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.819223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.820882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:43.833933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 924/1800] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:43.835819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:43.847700 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31cd0-2580-4435-90b2-d0eb9b4afc5a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:43.866119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:43.866119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:43.866119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:43.866119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] PATH_INFO: `/v3/projects/901e3d8e4a32418ea190b8d1eb216a80/users/8229d36feeb343f28d75eacacf7a717f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:43.878662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901e3d8e4a32418ea190b8d1eb216a80, user_id=8229d36feeb343f28d75eacacf7a717f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:43.882652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34f62a-2fef-4f64-ba58-ccb8e564d9e6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.022524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:44.029137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 925/1801] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:43 2022] PUT /identity/v3/projects/901e3d8e4a32418ea190b8d1eb216a80/users/8229d36feeb343f28d75eacacf7a717f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:44.048686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.050702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.051222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.054787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6988bf260544a81bbec9b1af8873824', 'request_id': 'req-2934bd70-01d1-474b-9845-ac4a4fa20dcf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:44.064366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6988bf260544a81bbec9b1af8873824', 'request_id': 'req-2934bd70-01d1-474b-9845-ac4a4fa20dcf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:44.081498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] MFA Rules not processed for user `8229d36feeb343f28d75eacacf7a717f`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:44.082758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:44.147412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 926/1802] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:44.151806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.158253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6988bf260544a81bbec9b1af8873824', 'request_id': 'req-2934bd70-01d1-474b-9845-ac4a4fa20dcf'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:44.171076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] The token cache is being invalidate because project f6988bf260544a81bbec9b1af8873824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:44.171370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:44.172294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2934bd70-01d1-474b-9845-ac4a4fa20dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:44.173950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 877/1803] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:43 2022] DELETE /identity/v3/projects/f6988bf260544a81bbec9b1af8873824 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:44.179423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab60928-a1c1-4de1-9df4-16729af38792 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.179790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab60928-a1c1-4de1-9df4-16729af38792 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.180125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab60928-a1c1-4de1-9df4-16729af38792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.201354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cab60928-a1c1-4de1-9df4-16729af38792 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:44.202797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cab60928-a1c1-4de1-9df4-16729af38792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:44.230951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde0750-bcee-4713-8610-59a9732923f2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.235816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.236996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.237476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.237934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.238727 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.240940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.269737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 878/1804] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:44.278465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.278749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.279019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.301792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:44.303010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:44.312002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 879/1805] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:44.316351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.325969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ed38a-1294-463f-a649-58b1a8ff48ad None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.330507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.331096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.331373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.331639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.332426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:44.332735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.334339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.345442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 880/1806] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:06:44.349510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.359125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317a96-8cb7-43a1-8c27-705d1252f053 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.363598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.364157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.364442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.364708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.365301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.367128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.405590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 927/1807] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:44.409466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.411125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:44.474834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896d228-fbd9-4652-a3e0-2f651a35949a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.480110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.480741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.481017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.481287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.481898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.483826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.526376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 881/1808] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/projects => generated 409 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:44.530382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.589860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090708a-3eab-4199-93af-4360cb481437 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.594501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.595108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.595388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.595660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.596325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.598243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.695555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 928/1809] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:44.699415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.708937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc943dab-7ac9-4ab7-99c0-ddd0e4663313 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.714119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.714707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.715012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.715284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.715921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.717610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.727438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 929/1810] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:44.731467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 882/1811] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:44 2022] POST /identity/v3/users => generated 305 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:44.736456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.736611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.746497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb230-a92f-4de1-aae0-f724bc268698 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.747077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a261504-3edb-43ad-b15d-0bdff459acb7 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.751600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.752222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.752506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.752636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.752781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.753423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.753533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.753877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.754218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] PATH_INFO: `/v3/projects/c4fd8f3908e2451ca95b46393834aa18/users/ad7bd4a750464650b993122b7aee239b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.755005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.766682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 930/1812] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:44.770752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.782398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17481d-7d10-49a6-a2b5-0d188e9284ab admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.787394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.792329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.792329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.792329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.792329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.792329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.793652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4fd8f3908e2451ca95b46393834aa18, user_id=ad7bd4a750464650b993122b7aee239b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.796634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.802114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 931/1813] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:44.807296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.817458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e1af-c042-4881-bc79-eb5625abf6a1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.827139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.827139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.827139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.827139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] PATH_INFO: `/v3/projects/402a4f02147e4add96804a9c0cd8d387/users/6d3388adb2c045929f50a56ed0dfa34e/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.843728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402a4f02147e4add96804a9c0cd8d387, user_id=6d3388adb2c045929f50a56ed0dfa34e, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.846445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c66d5-76f3-46b9-9cfe-9a81f854a529 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.892778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 883/1814] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:44 2022] PUT /identity/v3/projects/c4fd8f3908e2451ca95b46393834aa18/users/ad7bd4a750464650b993122b7aee239b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:44.903693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:44.970330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203cb93-0c68-4e9e-932c-70f82904e94d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:44.975093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:44.975705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:44.975985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:44.976283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:44.976928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:44.978890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:44.991966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 884/1815] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:44.992115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 932/1816] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:44 2022] PUT /identity/v3/projects/402a4f02147e4add96804a9c0cd8d387/users/6d3388adb2c045929f50a56ed0dfa34e/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:44.998715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.006151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa10bd2-4280-4235-b16c-72f73861d64f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.009458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.010047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.010681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.044374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] MFA Rules not processed for user `6d3388adb2c045929f50a56ed0dfa34e`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:45.045657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.073136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.073808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.074390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.074671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] PATH_INFO: `/v3/projects/c4fd8f3908e2451ca95b46393834aa18/users/ad7bd4a750464650b993122b7aee239b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.077575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4fd8f3908e2451ca95b46393834aa18, user_id=ad7bd4a750464650b993122b7aee239b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.081357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b73069-602a-4ce1-b776-3a6c7ff91230 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.097365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 885/1817] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:44 2022] PUT /identity/v3/projects/c4fd8f3908e2451ca95b46393834aa18/users/ad7bd4a750464650b993122b7aee239b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:45.112822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.113511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.114605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.151347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 933/1818] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:45.155325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] MFA Rules not processed for user `ad7bd4a750464650b993122b7aee239b`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:45.155960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.156773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.168594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e7162-6be8-413d-b1c5-8f7046b2a501 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.251797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.252449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.252744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.256948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.256948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.257184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 886/1819] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:45.257365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.279373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.279373 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.326938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef3b5-8afd-411b-8a33-a1ea8a34c48b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.331212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.331822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.332103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.332424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.344167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.345692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.357531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 887/1820] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:45.387719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.398617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b561632-11d9-4b44-8297-64a76df1aeb7 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.404836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.405574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.405856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.406240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] PATH_INFO: `/v3/users/8229d36feeb343f28d75eacacf7a717f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.407378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8229d36feeb343f28d75eacacf7a717f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.409453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.471667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:45.474210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 934/1821] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:45 2022] POST /identity/v3/projects => generated 407 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:45.478576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.551728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8302b-6a56-46b2-b045-f79f24d79fbe admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.557694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.558302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.558582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.558854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.559477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.561409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.703720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8229d36feeb343f28d75eacacf7a717f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:45.704404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8229d36feeb343f28d75eacacf7a717f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:45.711300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8229d36feeb343f28d75eacacf7a717f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:45.773375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 935/1822] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:45 2022] POST /identity/v3/users => generated 310 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:45.777957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.787464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b632-7ad1-4ac8-830c-66d24a54deb1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.792728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.793314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.793591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.793862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.794582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.796165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.803448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8229d36feeb343f28d75eacacf7a717f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:45.804029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0d846d2e-8f72-4bf5-bf03-23d4d54c33fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8229d36feeb343f28d75eacacf7a717f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:45.807176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 936/1823] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:45.814361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.816051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.822226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 888/1824] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:45 2022] DELETE /identity/v3/users/8229d36feeb343f28d75eacacf7a717f => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:45.826502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd297-6809-4208-a918-5337ce501e9c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.826795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd297-6809-4208-a918-5337ce501e9c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.827073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd297-6809-4208-a918-5337ce501e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.829509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b60-057e-4d8e-809c-563bd8a27c90 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.846337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-238bd297-6809-4208-a918-5337ce501e9c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:45.847587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-238bd297-6809-4208-a918-5337ce501e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.891564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.892263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.892544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.892814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] PATH_INFO: `/v3/projects/06e0942d069c4012b8c758b53d399f34/users/b8f92ae2b2844475a9df6ac23fe1b739/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:45.907960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e0942d069c4012b8c758b53d399f34, user_id=b8f92ae2b2844475a9df6ac23fe1b739, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:45.910763 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:45.911255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 889/1825] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:45.923672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bd297-6809-4208-a918-5337ce501e9c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:45.925147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-238bd297-6809-4208-a918-5337ce501e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:45.988165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bd297-6809-4208-a918-5337ce501e9c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:45.991713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:45.992357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:45.992633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:45.992915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.002384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:46.003821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:46.014202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 890/1826] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:46.061527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 937/1827] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:45 2022] PUT /identity/v3/projects/06e0942d069c4012b8c758b53d399f34/users/b8f92ae2b2844475a9df6ac23fe1b739/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:46.065497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:46.076410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c7625-f843-4829-b50f-51eedad2e725 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:46.135221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:46.135915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:46.136130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:46.136492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.137169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:46.138837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7235e-3476-46a6-934e-7a1547bec14c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:46.149082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 891/1828] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:46.154204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:46.164729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b8b1-9160-4188-ac51-01cebab4e80b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:46.170718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:46.171387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:46.171670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:46.172000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] PATH_INFO: `/v3/projects/06e0942d069c4012b8c758b53d399f34/users/b8f92ae2b2844475a9df6ac23fe1b739/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.173974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e0942d069c4012b8c758b53d399f34, user_id=b8f92ae2b2844475a9df6ac23fe1b739, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:46.176636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:46.197917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 938/1829] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:46 2022] PUT /identity/v3/projects/06e0942d069c4012b8c758b53d399f34/users/b8f92ae2b2844475a9df6ac23fe1b739/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:46.206996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:46.207291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:46.207573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.235496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] MFA Rules not processed for user `b8f92ae2b2844475a9df6ac23fe1b739`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:46.236827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:46.305918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 892/1830] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:46.316010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:46.318070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:46.385792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e43f4-0707-47e7-abd7-69c3a3aac18d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:46.390488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:46.391362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:46.395864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:46.395864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.403245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:46.404827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:46.424295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 939/1831] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:46 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:46.879778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:46.881219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:46.959183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6921a54-71c4-4ecd-a400-d7c3318aa7b7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:46.963849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:46.964517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:46.964795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:46.965057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] PATH_INFO: `/v3/projects/901e3d8e4a32418ea190b8d1eb216a80` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:46.966172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901e3d8e4a32418ea190b8d1eb216a80)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:46.968220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:47.042471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:47.051033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be515-c451-4ee9-802e-d35196b5d3a0 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:47.054748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:47.060567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:47.060567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:47.060567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:47.060567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:47.116696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:47.118715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:47.127437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 940/1832] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:47.151543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901e3d8e4a32418ea190b8d1eb216a80', 'request_id': 'req-50b4b424-aecd-48da-9faa-dadb71deb1d7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.157095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901e3d8e4a32418ea190b8d1eb216a80', 'request_id': 'req-50b4b424-aecd-48da-9faa-dadb71deb1d7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.261877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901e3d8e4a32418ea190b8d1eb216a80', 'request_id': 'req-50b4b424-aecd-48da-9faa-dadb71deb1d7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.274704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] The token cache is being invalidate because project 901e3d8e4a32418ea190b8d1eb216a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:47.275425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.279703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.279703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 893/1833] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:46 2022] DELETE /identity/v3/projects/901e3d8e4a32418ea190b8d1eb216a80 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:47.281776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:47.293435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555771b-57b6-4a0e-a005-befaff5b3c64 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:47.350588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:47.351221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:47.351500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:47.351774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] PATH_INFO: `/v3/users/ad7bd4a750464650b993122b7aee239b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:47.353869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7bd4a750464650b993122b7aee239b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:47.355767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:47.715021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7bd4a750464650b993122b7aee239b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.715608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7bd4a750464650b993122b7aee239b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.722220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7bd4a750464650b993122b7aee239b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.815310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7bd4a750464650b993122b7aee239b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.815967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7bd4a750464650b993122b7aee239b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:47.834605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 941/1834] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:47 2022] DELETE /identity/v3/users/ad7bd4a750464650b993122b7aee239b => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:48.330124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:48.337525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:48.339019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:48.370286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b424-aecd-48da-9faa-dadb71deb1d7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:48.433665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:48.434336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.434599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.437723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] PATH_INFO: `/v3/projects/c4fd8f3908e2451ca95b46393834aa18` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.437723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4fd8f3908e2451ca95b46393834aa18)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:48.438330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:48.439232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:48.441157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:48.456658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898055-05ab-4ba2-b013-c5dabafe3411 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:48.463846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:48.463846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.464256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.464672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] PATH_INFO: `/v3/users/7eb99be8312142a6adc144675472a1ab` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.466134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb99be8312142a6adc144675472a1ab)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:48.468335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:48.645425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4fd8f3908e2451ca95b46393834aa18', 'request_id': 'req-a1ce9980-273c-4493-8205-44b20d736d5a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.652192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4fd8f3908e2451ca95b46393834aa18', 'request_id': 'req-a1ce9980-273c-4493-8205-44b20d736d5a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.781347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4fd8f3908e2451ca95b46393834aa18', 'request_id': 'req-a1ce9980-273c-4493-8205-44b20d736d5a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.794993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] The token cache is being invalidate because project c4fd8f3908e2451ca95b46393834aa18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:48.799287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.799287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a1ce9980-273c-4493-8205-44b20d736d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.799287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 894/1835] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:48 2022] DELETE /identity/v3/projects/c4fd8f3908e2451ca95b46393834aa18 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:48.807671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a386-24a4-4595-ba74-4df050580df9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.808112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a386-24a4-4595-ba74-4df050580df9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.808508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067a386-24a4-4595-ba74-4df050580df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.839874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6067a386-24a4-4595-ba74-4df050580df9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:48.841186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6067a386-24a4-4595-ba74-4df050580df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:48.845011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb99be8312142a6adc144675472a1ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.845011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb99be8312142a6adc144675472a1ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.850995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb99be8312142a6adc144675472a1ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.900448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 895/1836] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:48.908132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.908442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.908721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.928877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:48.930131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:48.942081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 896/1837] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:48.943837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb99be8312142a6adc144675472a1ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.944631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:48.945399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-abf56ff7-eedc-4453-b4ae-933c9f3f6b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb99be8312142a6adc144675472a1ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:48.965414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a01a23-13ea-4e2c-a350-cd98e42da804 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:48.971027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:48.971767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.972008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 942/1838] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:48 2022] DELETE /identity/v3/users/7eb99be8312142a6adc144675472a1ab => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:48.972287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.972581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.973244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:48.973554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:48.975231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:48.976598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:48.977053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:48.978041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:48.990348 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 897/1839] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:48.999638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:49.001146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.001294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:49.003121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:49.017122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848582e-b890-401b-94c6-494001525ba5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.070354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 943/1840] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:49.076452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.077068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.077346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.077615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.078256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.082045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.083075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.086272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:49.147981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8b1b7-b3a2-4555-8979-18837c1d93b6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.152679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.153291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.153572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.153843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.163168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.164699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.174979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 944/1841] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:49.240428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:49.250038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 898/1842] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:49 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:06:49.255795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.266882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa6940-fba6-4e7d-a10c-2b1f67786cca service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.322185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.322185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.322471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.322742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.323358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.325276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.516073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 945/1843] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:49 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:49.517520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.543815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de4640-0474-4714-aacd-fdbc3d907233 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.562658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.562658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.562658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.562658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.563201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.564084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.600635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 899/1844] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:49.600727 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.608279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eacc57-19ad-4f72-8dc7-e5afb7889643 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.613832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.615256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.615749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.616263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.617281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.620916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.637652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 946/1845] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:49.642529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.659954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02190df4-5a1b-4c76-a807-eab23e8e8637 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.666396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.667731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.668112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.668561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] PATH_INFO: `/v3/projects/cae61024c9584fc4bf1d431ff9865262/users/8e5e5ec47a054feeba3d32947d445915/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.732509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae61024c9584fc4bf1d431ff9865262, user_id=8e5e5ec47a054feeba3d32947d445915, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.739312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8068f2-98b4-49ac-a12d-4da13824abcc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.825352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:49.827330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:49.853099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33fb85-a626-47a2-a71a-60ccbaeb5f4b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:49.874608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 900/1846] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:49 2022] PUT /identity/v3/projects/cae61024c9584fc4bf1d431ff9865262/users/8e5e5ec47a054feeba3d32947d445915/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:49.884278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:49.892410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.892410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.892410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] PATH_INFO: `/v3/projects/e759fac3354b4a488ab40788db8db563` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.892888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:49.892888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:49.892888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:49.893074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e759fac3354b4a488ab40788db8db563)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:49.893074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:49.958360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] MFA Rules not processed for user `8e5e5ec47a054feeba3d32947d445915`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:49.959788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:50.095688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 901/1847] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:50.101093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.106141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e759fac3354b4a488ab40788db8db563', 'request_id': 'req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.170796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e759fac3354b4a488ab40788db8db563', 'request_id': 'req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.186232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e759fac3354b4a488ab40788db8db563', 'request_id': 'req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.205484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f2cd-0390-4f8d-bcd8-803b20dca472 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.275359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.275984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.276283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.276558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.277428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.279354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.366951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] The token cache is being invalidate because project e759fac3354b4a488ab40788db8db563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:50.366951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.366951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.367521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 947/1848] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:49 2022] DELETE /identity/v3/projects/e759fac3354b4a488ab40788db8db563 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:50.372584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.374046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:50.388333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2e101-9bf6-4b48-aeff-bc2c3e1439f7 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.392895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.393536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.393810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.394085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] PATH_INFO: `/v3/users/f0d467533cc24f75ab5bafb0672a369a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.395203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d467533cc24f75ab5bafb0672a369a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.397189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.429844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 902/1849] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:50 2022] POST /identity/v3/projects => generated 383 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:50.431391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.433051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:50.498465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a41b-7684-474e-9fbb-67c95e8999ec admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.503148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.503757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.504032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.504324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.504951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.506871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.621304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 903/1850] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:50 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:50.625032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.631186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d467533cc24f75ab5bafb0672a369a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.631846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d467533cc24f75ab5bafb0672a369a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.636551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45fdb3-eeb0-452b-929d-0e2b51e5ea5a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.639686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d467533cc24f75ab5bafb0672a369a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.641560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.642511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.642874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.643211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.643923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.645841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.657105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 904/1851] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:06:50.661124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.670893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe43c26-7183-4c53-915a-58f6965ca9c2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.675504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.676167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.676465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.676756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] PATH_INFO: `/v3/projects/0b29657d33c7487dbbae96e00eb486da/users/2552eeca0bd04a0a87f06e15baefa549/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.691170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b29657d33c7487dbbae96e00eb486da, user_id=2552eeca0bd04a0a87f06e15baefa549, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.696851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.763824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d467533cc24f75ab5bafb0672a369a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.764492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d467533cc24f75ab5bafb0672a369a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:50.793918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 948/1852] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:50 2022] DELETE /identity/v3/users/f0d467533cc24f75ab5bafb0672a369a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:50.838758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 905/1853] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:50 2022] PUT /identity/v3/projects/0b29657d33c7487dbbae96e00eb486da/users/2552eeca0bd04a0a87f06e15baefa549/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:50.842824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:50.844246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:50.913997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48422-b648-4050-9a30-51eb248645c2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:50.976408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:50.977039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:50.977351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:50.977649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:50.978454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:50.980140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff625df-51f8-4a3c-9019-25547b2f7d19 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:50.992410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 949/1854] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:51.005817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:51.017665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da7d3d-2658-4672-9ce3-3ee2e0290639 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:51.022321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:51.022990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.023295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.023556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] PATH_INFO: `/v3/projects/0b29657d33c7487dbbae96e00eb486da/users/2552eeca0bd04a0a87f06e15baefa549/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.026140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b29657d33c7487dbbae96e00eb486da, user_id=2552eeca0bd04a0a87f06e15baefa549, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:51.028985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:51.040413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 906/1855] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:51 2022] PUT /identity/v3/projects/0b29657d33c7487dbbae96e00eb486da/users/2552eeca0bd04a0a87f06e15baefa549/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:51.050686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.050979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.051258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.083830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] MFA Rules not processed for user `2552eeca0bd04a0a87f06e15baefa549`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:51.085114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:51.157333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 950/1856] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:51.175322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:51.177155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:51.240692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f216e93-6010-4504-bf50-cb083976537a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:51.245815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:51.246416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.246764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.248273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.258210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:51.260652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:51.273314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 907/1857] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:51.408055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:51.427947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0b057-7ead-4216-b32a-75af8939d7e3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:51.432660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:51.433451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.433880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.434287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.436849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:51.502229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:51.503766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ff0b2-be1d-49ab-9ec8-e3359a237047 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:51.513399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 951/1858] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:51.625941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:51.627453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:51.695591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c05e7-86c2-4322-aa0b-550267e3ac91 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:51.700837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:51.704789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.704789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.704789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] PATH_INFO: `/v3/projects/7a774755645e46b3838237a5cd1f1fb6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.704789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a774755645e46b3838237a5cd1f1fb6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:51.705135 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:51.818237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a774755645e46b3838237a5cd1f1fb6', 'request_id': 'req-47c0bdf3-875b-4545-8bd1-8616f128d98f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:51.824204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a774755645e46b3838237a5cd1f1fb6', 'request_id': 'req-47c0bdf3-875b-4545-8bd1-8616f128d98f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:51.909093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a774755645e46b3838237a5cd1f1fb6', 'request_id': 'req-47c0bdf3-875b-4545-8bd1-8616f128d98f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:51.921106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] The token cache is being invalidate because project 7a774755645e46b3838237a5cd1f1fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:51.921398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:51.922124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-47c0bdf3-875b-4545-8bd1-8616f128d98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:51.923628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 908/1859] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:51 2022] DELETE /identity/v3/projects/7a774755645e46b3838237a5cd1f1fb6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:51.928596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392092-ecce-435e-a6c8-812d02585817 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:51.929075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392092-ecce-435e-a6c8-812d02585817 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:51.929166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392092-ecce-435e-a6c8-812d02585817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:51.950530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-66392092-ecce-435e-a6c8-812d02585817 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:51.951658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-66392092-ecce-435e-a6c8-812d02585817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:52.010966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 952/1860] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:52.015164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:52.018756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.019039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.019315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.039056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:52.040416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:52.048114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 909/1861] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:52.052268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66392092-ecce-435e-a6c8-812d02585817 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.062858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66392092-ecce-435e-a6c8-812d02585817 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.067406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.068007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.068323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.068598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.069245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:52.069552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.071796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.083148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 953/1862] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:06:52.087660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.096911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5585e58-36b2-4959-97bb-7a01dcceecf9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.101663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.102421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.102702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.102981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.103581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.105547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.209034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 910/1863] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:52.212185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.223884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f0e-0d17-4d66-a6d8-5e8e0d1d3551 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.286911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.287582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.287876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.288175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.288839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.291699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.407139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 954/1864] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:52.411300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.421648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988529f9-5242-479a-8dd1-be341dfa1560 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.426321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.426928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.427208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.427482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.428184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.429804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.440332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 911/1865] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:52.444392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.456222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3b0a-768e-4f35-a7c9-60b65b9e10d4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.470216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.470216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.470216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.470216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.470758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.470758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f5bc-c7f3-445f-a981-d4d9e3558e06 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.481455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 955/1866] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:52.485770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.497311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13e81-ed9c-4458-adf1-0b08f2bef0bf admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.502803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.503474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.503760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.504039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] PATH_INFO: `/v3/projects/222429245df64cf3949bf111c97eccd0/users/ac2f86f2ab144249be40efab9a01bd03/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.521190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222429245df64cf3949bf111c97eccd0, user_id=ac2f86f2ab144249be40efab9a01bd03, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.524037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.647190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 912/1867] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:52 2022] PUT /identity/v3/projects/222429245df64cf3949bf111c97eccd0/users/ac2f86f2ab144249be40efab9a01bd03/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:52.662784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.663074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.663345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.692241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] MFA Rules not processed for user `ac2f86f2ab144249be40efab9a01bd03`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:52.693462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:52.721090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:52.760171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 956/1868] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:52.764681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.827651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df92fc-a268-4415-a63f-540eb83e0780 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:52.832568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:52.833175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:52.833469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:52.833728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:52.834356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:52.836287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:52.965381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 913/1869] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:52.969384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:52.980289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219dc82-e36c-4341-a624-c34ee7cb04cc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.037896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.038515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.038793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.039065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.039711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.041876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.192701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 957/1870] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:52 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:53.197443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:53.207694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b79f37-3880-4084-b873-6a03efd8dc3c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.212404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.213034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.213326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.213614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.214277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.215872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.226364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 914/1871] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:53.231280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:53.241239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb28d65-fa79-4ee7-8e86-387bd149d18c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.246540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.247190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.247481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.247757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] PATH_INFO: `/v3/projects/ccd5692e100445b8b84e9c7998d84d25/users/e79885b82f2d4e6a9cbd359da6efaf9a/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.264424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd5692e100445b8b84e9c7998d84d25, user_id=e79885b82f2d4e6a9cbd359da6efaf9a, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.267192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.464493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 958/1872] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:53 2022] PUT /identity/v3/projects/ccd5692e100445b8b84e9c7998d84d25/users/e79885b82f2d4e6a9cbd359da6efaf9a/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:53.468899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:53.480883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c2e94-4ff0-4097-91a4-2a8bd6dfa68d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.539110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.540430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.540720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.540999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.541650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.543305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb42d76-e0b5-4b14-a637-c8d316bcbee4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.553847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 915/1873] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:06:53.558540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:53.570020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbccb4-5fd8-44d8-ba4b-fd6a36b7a7ae admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.576403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.577926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.578207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.578483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] PATH_INFO: `/v3/projects/ccd5692e100445b8b84e9c7998d84d25/users/e79885b82f2d4e6a9cbd359da6efaf9a/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.580750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd5692e100445b8b84e9c7998d84d25, user_id=e79885b82f2d4e6a9cbd359da6efaf9a, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.583972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.604201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 959/1874] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:06:53 2022] PUT /identity/v3/projects/ccd5692e100445b8b84e9c7998d84d25/users/e79885b82f2d4e6a9cbd359da6efaf9a/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:53.609471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c176e-cd99-4b1f-87b6-c8a64b807cd5 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.609877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c176e-cd99-4b1f-87b6-c8a64b807cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.610224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c176e-cd99-4b1f-87b6-c8a64b807cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.654057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e60c176e-cd99-4b1f-87b6-c8a64b807cd5 None None] MFA Rules not processed for user `e79885b82f2d4e6a9cbd359da6efaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:53.657174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e60c176e-cd99-4b1f-87b6-c8a64b807cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:53.783692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 916/1875] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:53.791843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:53.793881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:53.925269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0da429-ae20-48b9-9492-8a07c74f13b8 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:53.938872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:53.938872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:53.938872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:53.939146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:53.950411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:53.951999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2815-8195-48c2-bbfa-fefab0fd7ba1 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:53.964415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 960/1876] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:54.150037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:54.150749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:54.151522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:54.154592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 917/1877] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:06:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:06:54.184912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:54.184912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:54.184912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:54.203509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:54.206115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:54.288857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:54.292339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:54.306537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 961/1878] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:06:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:54.314428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750fdc-5935-4b11-a8b1-5028722a6f25 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:54.318457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:54.321174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:54.321174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:54.321174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:54.324922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:54.326402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:54.360111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3f3f6-a731-413e-98e0-55bded112a4c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:54.378380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:54.379392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:54.379869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:54.380370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:54.392267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:54.393867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:54.399240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:54.401150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581385c3-22b2-40a6-a5ff-fb806ca862e3 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:54.405898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 962/1879] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:54.424533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 918/1880] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:06:54.563994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:55.199217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:55.739493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:55.809266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f47b-e4d8-459f-8e5a-f31ea63c9ba2 service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:55.813908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:55.814537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:55.814814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:55.815083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] PATH_INFO: `/v3/users/ac2f86f2ab144249be40efab9a01bd03` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:55.816203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2f86f2ab144249be40efab9a01bd03)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:55.818137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:56.079711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2f86f2ab144249be40efab9a01bd03'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.080601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2f86f2ab144249be40efab9a01bd03'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.091551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2f86f2ab144249be40efab9a01bd03'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.167549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2f86f2ab144249be40efab9a01bd03'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.168294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2f86f2ab144249be40efab9a01bd03'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.187551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 963/1881] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:55 2022] DELETE /identity/v3/users/ac2f86f2ab144249be40efab9a01bd03 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:56.192508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:56.192880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:56.193160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:56.213891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:56.215099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:56.273636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 919/1882] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:56.283994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:56.285431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:56.314023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:56.315469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:56.328486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896febc-ae2e-4f4d-978b-e88d99460965 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:56.335453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:56.336392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:56.337244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:56.337692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] PATH_INFO: `/v3/users/8e5e5ec47a054feeba3d32947d445915` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:56.339144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5e5ec47a054feeba3d32947d445915)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:56.341209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:56.351612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bffed-a4ce-4c01-868e-9baf1f7888e4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:56.355296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:56.355908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:56.356223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:56.356488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:56.366090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:56.367514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571f90-4ddd-4f7c-bd0a-5e422bb753b8 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:56.377686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 964/1883] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:56.639949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:56.653794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5e5ec47a054feeba3d32947d445915'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.658095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5e5ec47a054feeba3d32947d445915'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.661743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5e5ec47a054feeba3d32947d445915'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.806295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5e5ec47a054feeba3d32947d445915'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.808019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5e5ec47a054feeba3d32947d445915'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:56.840977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 920/1884] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:56 2022] DELETE /identity/v3/users/8e5e5ec47a054feeba3d32947d445915 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:56.845460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:56.846185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:56.846418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:56.876703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:56.878815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:56.966842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 965/1885] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:56.976746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:56.978533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:57.066591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7644956-1e26-42ae-b65d-533b6ea9b6ce admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:57.071153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:57.071918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:57.072814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:57.072926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:57.087279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:57.088443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:57.090253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:57.091711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:57.113744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1865c-98c4-4a11-9787-08631c5ebccf None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:57.123519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:57.125558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 921/1886] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:06:57.129000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:57.129000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:57.129000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] PATH_INFO: `/v3/projects/222429245df64cf3949bf111c97eccd0` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:57.129000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=222429245df64cf3949bf111c97eccd0)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:57.131620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:57.478871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222429245df64cf3949bf111c97eccd0', 'request_id': 'req-da62cab3-391c-4334-9c4b-984ce768d46e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:57.484467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222429245df64cf3949bf111c97eccd0', 'request_id': 'req-da62cab3-391c-4334-9c4b-984ce768d46e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:57.491012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222429245df64cf3949bf111c97eccd0', 'request_id': 'req-da62cab3-391c-4334-9c4b-984ce768d46e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:57.592522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:57.684488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] The token cache is being invalidate because project 222429245df64cf3949bf111c97eccd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:57.684842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:57.685528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:57.687327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 966/1887] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:57 2022] DELETE /identity/v3/projects/222429245df64cf3949bf111c97eccd0 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:57.691375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:57.702446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fad67-fade-436c-befe-3bd37f4e8906 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:57.764191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:57.764785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:57.765069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:57.765344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] PATH_INFO: `/v3/users/e79885b82f2d4e6a9cbd359da6efaf9a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:57.766497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79885b82f2d4e6a9cbd359da6efaf9a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:57.768528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:58.072613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79885b82f2d4e6a9cbd359da6efaf9a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.074121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79885b82f2d4e6a9cbd359da6efaf9a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.096608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79885b82f2d4e6a9cbd359da6efaf9a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.138688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:58.141067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:58.155967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62cab3-391c-4334-9c4b-984ce768d46e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:58.164062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:58.165506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:58.165789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:58.166068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] PATH_INFO: `/v3/projects/cae61024c9584fc4bf1d431ff9865262` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:58.167231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae61024c9584fc4bf1d431ff9865262)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:58.169455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:58.281503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79885b82f2d4e6a9cbd359da6efaf9a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.282112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79885b82f2d4e6a9cbd359da6efaf9a'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.313768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 922/1888] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:57 2022] DELETE /identity/v3/users/e79885b82f2d4e6a9cbd359da6efaf9a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:58.351034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae61024c9584fc4bf1d431ff9865262', 'request_id': 'req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.359211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae61024c9584fc4bf1d431ff9865262', 'request_id': 'req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.369890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae61024c9584fc4bf1d431ff9865262', 'request_id': 'req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.507025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:06:58.564504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] The token cache is being invalidate because project cae61024c9584fc4bf1d431ff9865262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:58.564793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.567612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:58.567893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 967/1889] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:58 2022] DELETE /identity/v3/projects/cae61024c9584fc4bf1d431ff9865262 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:06:58.573189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:58.574662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:58.697869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aed772-201a-46ba-a84d-dfd274beca2d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:58.779835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:58.781416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:58.782136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:58.782647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] PATH_INFO: `/v3/users/2552eeca0bd04a0a87f06e15baefa549` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:58.789089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2552eeca0bd04a0a87f06e15baefa549)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:58.793191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.041153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.043071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.059802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc9d0-8fd2-4dbd-afdf-48b3c05c7d1f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.073564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.074309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.074751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.075258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] PATH_INFO: `/v3/projects/ccd5692e100445b8b84e9c7998d84d25` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.077056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd5692e100445b8b84e9c7998d84d25)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.079771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.123584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2552eeca0bd04a0a87f06e15baefa549'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.125620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2552eeca0bd04a0a87f06e15baefa549'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.150900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2552eeca0bd04a0a87f06e15baefa549'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.259029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd5692e100445b8b84e9c7998d84d25', 'request_id': 'req-cd3b70a0-5823-4a45-aca1-af9a4b77891a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.264984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd5692e100445b8b84e9c7998d84d25', 'request_id': 'req-cd3b70a0-5823-4a45-aca1-af9a4b77891a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.271455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd5692e100445b8b84e9c7998d84d25', 'request_id': 'req-cd3b70a0-5823-4a45-aca1-af9a4b77891a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.272164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2552eeca0bd04a0a87f06e15baefa549'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.272796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2552eeca0bd04a0a87f06e15baefa549'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.292509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 923/1890] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:58 2022] DELETE /identity/v3/users/2552eeca0bd04a0a87f06e15baefa549 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:59.293386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.294797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.363310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa34b9-3fa6-4a45-a581-1f27913e8552 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.369169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.369842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.370120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.370394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] PATH_INFO: `/v3/users/6d3388adb2c045929f50a56ed0dfa34e` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.371515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3388adb2c045929f50a56ed0dfa34e)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.373497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.422834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] The token cache is being invalidate because project ccd5692e100445b8b84e9c7998d84d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:06:59.423126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.423824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cd3b70a0-5823-4a45-aca1-af9a4b77891a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.425401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 968/1891] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:59 2022] DELETE /identity/v3/projects/ccd5692e100445b8b84e9c7998d84d25 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:59.431771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faad81-2e9c-4ac0-8f4f-02a836a3a242 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.432045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faad81-2e9c-4ac0-8f4f-02a836a3a242 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.432352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faad81-2e9c-4ac0-8f4f-02a836a3a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.452317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-11faad81-2e9c-4ac0-8f4f-02a836a3a242 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:59.456630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-11faad81-2e9c-4ac0-8f4f-02a836a3a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.469413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 969/1892] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:59.477031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.477573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.477968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.497433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:59.498627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.506519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 970/1893] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:06:59.511825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.521457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b430426-f712-4077-b8d4-bb184e0dd867 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.533443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.534974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.535466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.535945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.537028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:06:59.537717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.539832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.565385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 971/1894] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:06:59.566534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.596255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327334ef-e1b5-4a2e-93c3-7906c593e76c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.605817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.606935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.607368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.607801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.609084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.612354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.648451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3388adb2c045929f50a56ed0dfa34e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.649747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3388adb2c045929f50a56ed0dfa34e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.657266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3388adb2c045929f50a56ed0dfa34e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.733708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 972/1895] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:06:59 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:59.734749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.752442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3388adb2c045929f50a56ed0dfa34e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.753399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-37bac17d-90f5-44ce-9813-82fba008340c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3388adb2c045929f50a56ed0dfa34e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:06:59.774364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 924/1896] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:06:59 2022] DELETE /identity/v3/users/6d3388adb2c045929f50a56ed0dfa34e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:06:59.779179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.779616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.779901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.801358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:06:59.802580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.812177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5477-31a2-4a90-9cec-3d8c8140d221 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.883534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 925/1897] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:06:59.883534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.889173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:06:59.891552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.892298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.892583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.892860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.893503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.895417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.902789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b2506-7a6c-4860-8c52-7011727b7cc4 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:06:59.909975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:06:59.911274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:06:59.911274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:06:59.911519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] PATH_INFO: `/v3/projects/0b29657d33c7487dbbae96e00eb486da` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:06:59.912764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b29657d33c7487dbbae96e00eb486da)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:06:59.916739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:06:59.990349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 973/1898] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:06:59 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:06:59.991262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:06:59.993756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.065048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b29657d33c7487dbbae96e00eb486da', 'request_id': 'req-ec0dfcdb-64f1-41c7-8809-98302313f3f7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:00.065258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0804-bc9f-481c-858c-fb711f38948b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.071121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b29657d33c7487dbbae96e00eb486da', 'request_id': 'req-ec0dfcdb-64f1-41c7-8809-98302313f3f7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:00.121162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.121795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.122096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.122448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.178245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b29657d33c7487dbbae96e00eb486da', 'request_id': 'req-ec0dfcdb-64f1-41c7-8809-98302313f3f7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:00.191178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] The token cache is being invalidate because project 0b29657d33c7487dbbae96e00eb486da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:00.191473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:00.192194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:00.194020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 926/1899] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:06:59 2022] DELETE /identity/v3/projects/0b29657d33c7487dbbae96e00eb486da => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:00.194873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.196555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.224639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.225388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7066a-fb27-4add-b147-e390cb666f9b service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.240869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 974/1900] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:00.243034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635eea-a562-4d5a-8b21-e7f26f661ed3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.243247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635eea-a562-4d5a-8b21-e7f26f661ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.243625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635eea-a562-4d5a-8b21-e7f26f661ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.268056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-aa635eea-a562-4d5a-8b21-e7f26f661ed3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:00.269371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aa635eea-a562-4d5a-8b21-e7f26f661ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.283107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 975/1901] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:00.293103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.293393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.293666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.326308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dfcdb-64f1-41c7-8809-98302313f3f7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.333461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.334945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.335412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.336946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.338073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.340101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.352286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:00.355416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.356249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 927/1902] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:00.359910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.374864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 976/1903] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:00.380583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.390948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda795c-917d-4e06-9caf-91e3ad81c5da admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.402441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.403390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.403851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.404803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.405245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.405455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78b2e6-c764-4d44-9d95-c31df8cbce95 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.407328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.418175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.425793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.425793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.425793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.425793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:00.425793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.426269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 928/1904] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:00.426391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.427585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.440698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 977/1905] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:00.445670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.458080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8aac7-a39e-40a7-b294-3810c733a579 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.462663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64d410-f2b2-4ab7-9825-2ef73f0bc286 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.464098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.465231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.465669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.466144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.467129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.470090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.494721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.494721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.494721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.494721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] PATH_INFO: `/v3/projects/31b060632a774671af1631f203c763c2/users/3cc219984a5942b1bd39ad2265bbe85e/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.504594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b060632a774671af1631f203c763c2, user_id=3cc219984a5942b1bd39ad2265bbe85e, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.507653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3046-21b5-4c25-a379-011f00c81603 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.713472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 978/1906] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:00 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:00.714036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 929/1907] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:00 2022] PUT /identity/v3/projects/31b060632a774671af1631f203c763c2/users/3cc219984a5942b1bd39ad2265bbe85e/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:00.717241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.724974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f740b-4831-46e5-912e-05557864aee6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.725400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f740b-4831-46e5-912e-05557864aee6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.725880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f740b-4831-46e5-912e-05557864aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.779757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-722f740b-4831-46e5-912e-05557864aee6 None None] MFA Rules not processed for user `3cc219984a5942b1bd39ad2265bbe85e`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:00.783673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-722f740b-4831-46e5-912e-05557864aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.808308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a59f9-3680-4887-afe3-7da71a07dd8a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.816856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.817675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.818113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.818484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.819209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.821908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.870977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 930/1908] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:00.882625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f740b-4831-46e5-912e-05557864aee6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.884734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-722f740b-4831-46e5-912e-05557864aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:00.961108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 979/1909] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:00 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:00.961846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f740b-4831-46e5-912e-05557864aee6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.962303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.965561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.966169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.966451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.966789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.972052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbbe93-0f62-4fac-bed6-096b170cadbd admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:00.975361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.976804 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:00.976931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.977399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:00.977682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:00.977958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:00.978598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:00.980260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:00.985903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 931/1910] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:00.994158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 980/1911] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:00.995555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:00.999318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.003915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:01.018701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753be582-afd9-402c-9fed-d9e157a826a6 service glance] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.024998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.025841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.026165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.026536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.027286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.029258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.032916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ef41f-6cae-4372-82fd-c0befaf811fb admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.038255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.038900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.039199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.039581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] PATH_INFO: `/v3/projects/402a4f02147e4add96804a9c0cd8d387` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.041048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402a4f02147e4add96804a9c0cd8d387)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.042912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.047600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 932/1912] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:01.048669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.061916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a76c3-5f0a-42ee-95ee-b1399f5dcecb admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.066974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.067768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.068274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.068618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] PATH_INFO: `/v3/projects/918892c7ce41473d8dce5aa383727278/users/0f9977641a394011bfaa1bb0d52678b4/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.085454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918892c7ce41473d8dce5aa383727278, user_id=0f9977641a394011bfaa1bb0d52678b4, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.088690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62445d3d-8900-4347-8113-87d25a36d448 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.209493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 933/1913] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:01 2022] PUT /identity/v3/projects/918892c7ce41473d8dce5aa383727278/users/0f9977641a394011bfaa1bb0d52678b4/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:01.220319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.220707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.221107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.240659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402a4f02147e4add96804a9c0cd8d387', 'request_id': 'req-48cf4d00-05c4-4451-91c5-3add7afc1379'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.256530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402a4f02147e4add96804a9c0cd8d387', 'request_id': 'req-48cf4d00-05c4-4451-91c5-3add7afc1379'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.269578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] MFA Rules not processed for user `0f9977641a394011bfaa1bb0d52678b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:01.269670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402a4f02147e4add96804a9c0cd8d387', 'request_id': 'req-48cf4d00-05c4-4451-91c5-3add7afc1379'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.270081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:01.358452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 934/1914] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:01.362859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.397101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] The token cache is being invalidate because project 402a4f02147e4add96804a9c0cd8d387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:01.397581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.398423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.400248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 981/1915] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:00 2022] DELETE /identity/v3/projects/402a4f02147e4add96804a9c0cd8d387 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:01.404113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.406865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:01.470852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83633e10-bb9b-4590-9c06-dedc4b4ff6b9 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.482059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.482926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.483079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.483350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.483992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.486245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.497262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4d00-05c4-4451-91c5-3add7afc1379 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.503162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.503809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.504087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.504415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] PATH_INFO: `/v3/users/b8f92ae2b2844475a9df6ac23fe1b739` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.505544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f92ae2b2844475a9df6ac23fe1b739)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.507806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.603351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 935/1916] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:01 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:01.604551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.606603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:01.689966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:01.709995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d916-7148-4b89-93c0-01f70fa12506 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.716891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.717515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.717793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.718063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.718699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.720688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.784657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f92ae2b2844475a9df6ac23fe1b739'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.785934 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f92ae2b2844475a9df6ac23fe1b739'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.794084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f92ae2b2844475a9df6ac23fe1b739'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.908614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 936/1917] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:01 2022] POST /identity/v3/users => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:01.912368 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.922979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a232372-9df2-4c91-ad52-43932e1f8f2c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.927716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.928396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.928682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.928957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.929607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.933177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:01.944841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 937/1918] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:01.949436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:01.959073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484c009-b8c4-450b-b987-62b062b67d02 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:01.963864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:01.964537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:01.964821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:01.965098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] PATH_INFO: `/v3/projects/2eef366e93fc4519b39c11ad66b059ff/users/e06cc018f16145baa47ee565cbe1aa5b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:01.978652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f92ae2b2844475a9df6ac23fe1b739'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.979658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f92ae2b2844475a9df6ac23fe1b739'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:01.987168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eef366e93fc4519b39c11ad66b059ff, user_id=e06cc018f16145baa47ee565cbe1aa5b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:01.990105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:02.000337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 982/1919] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:01 2022] DELETE /identity/v3/users/b8f92ae2b2844475a9df6ac23fe1b739 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:02.127481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 938/1920] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:01 2022] PUT /identity/v3/projects/2eef366e93fc4519b39c11ad66b059ff/users/e06cc018f16145baa47ee565cbe1aa5b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:02.131821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:02.135131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:02.203998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ebc68-dfb4-4ccf-bcd0-43bdd2f8b1b0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:02.281012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:02.284053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:02.284762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.290545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.290545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.290545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:02.290545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794f20b-416c-4888-9488-a8c92f674c66 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:02.308656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:02.309736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 983/1921] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:02.321216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25982f-61dc-47af-b18f-49156d9c1583 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:02.327245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:02.328026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.328529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.328820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] PATH_INFO: `/v3/projects/2eef366e93fc4519b39c11ad66b059ff/users/e06cc018f16145baa47ee565cbe1aa5b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.331031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eef366e93fc4519b39c11ad66b059ff, user_id=e06cc018f16145baa47ee565cbe1aa5b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:02.333883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:02.348468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 939/1922] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:02 2022] PUT /identity/v3/projects/2eef366e93fc4519b39c11ad66b059ff/users/e06cc018f16145baa47ee565cbe1aa5b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:02.358330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.358631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.359077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.392339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] MFA Rules not processed for user `e06cc018f16145baa47ee565cbe1aa5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:02.394135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:02.468450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 984/1923] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:02.479815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:02.481645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:02.545836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68bd0f-51aa-445a-9df6-f10b505a8c9b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:02.548830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:02.549288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:02.549617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.550076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.550327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.550796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:02.563498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:02.565225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da6c4d-6667-4aaf-b96e-c0f4118c245c service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:02.583596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 940/1924] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:02.653018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21725c-c267-4ddb-ae0d-b85a57b064bd None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:02.657614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:02.658252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.658535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.658811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] PATH_INFO: `/v3/projects/06e0942d069c4012b8c758b53d399f34` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.659957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e0942d069c4012b8c758b53d399f34)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:02.661912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:02.806154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e0942d069c4012b8c758b53d399f34', 'request_id': 'req-473bee63-37f1-4c42-9507-88d2788aa1fa'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:02.812386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e0942d069c4012b8c758b53d399f34', 'request_id': 'req-473bee63-37f1-4c42-9507-88d2788aa1fa'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:02.819480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e0942d069c4012b8c758b53d399f34', 'request_id': 'req-473bee63-37f1-4c42-9507-88d2788aa1fa'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:02.913966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] The token cache is being invalidate because project 06e0942d069c4012b8c758b53d399f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:02.919060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:02.919060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-473bee63-37f1-4c42-9507-88d2788aa1fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:02.919060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 985/1925] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:02 2022] DELETE /identity/v3/projects/06e0942d069c4012b8c758b53d399f34 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:02.922851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:02.923747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:02.924028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:02.948347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:02.949596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:03.026966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 941/1926] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:03.034930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.035212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.035506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.056822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:03.058023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:03.066322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 986/1927] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:03.070772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.081606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005cd13-d13a-4596-9378-b3e6098137ad None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.089847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.090768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.091544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.092136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.093121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:03.093681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.095603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.110114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 942/1928] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:03.114729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.129094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16779f-3541-4a4a-a08e-e57aab8197d0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.156725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.159105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.159960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.160274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.167813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.167813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.341630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 987/1929] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:03 2022] POST /identity/v3/projects => generated 367 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:03.342750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.360282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22df5a-6fd1-408f-885e-0d02f5c6c2ea admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.454369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.454928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.455210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.455490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.456132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.458276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.629056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:03.637363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 943/1930] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:03 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:03.641235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.652093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6a9a-99cb-4d0e-ae13-7275b2c78738 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.656865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.657380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.657661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.657935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.658638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.660310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.677577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.678030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 988/1931] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:03.688671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce3e36-fdf4-44b7-b926-b7aaad4eb108 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.693778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.694520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.694916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.695288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.696781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.698496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5f7c7-92d1-42a7-87e9-ac648e36fedd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.714183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 944/1932] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:03.718059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:03.729870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5677a3-8539-4255-b649-d09090f2a895 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:03.734666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:03.741958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.741958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.741958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] PATH_INFO: `/v3/projects/e1ab8de906764692b7ff89614976486f/users/1ded9368eef941c7b8c6646d71d55e29/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.753413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ab8de906764692b7ff89614976486f, user_id=1ded9368eef941c7b8c6646d71d55e29, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:03.756102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:03.864071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 989/1933] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:03 2022] PUT /identity/v3/projects/e1ab8de906764692b7ff89614976486f/users/1ded9368eef941c7b8c6646d71d55e29/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:03.874888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:03.875179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:03.875485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:03.932683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] MFA Rules not processed for user `1ded9368eef941c7b8c6646d71d55e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:03.934224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:04.046956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 945/1934] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:04.051787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.171285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b00d2-1c3a-495c-89da-a893d5bc0396 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.178475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.179099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.179379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.179653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.180332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.182350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.356915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 990/1935] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:04 2022] POST /identity/v3/projects => generated 365 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:04.363437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.376581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14897d-6a24-4ba5-a664-b937c14048fc None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.454009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.454872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.455351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.455810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.456692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.458633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.577026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.580595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:04.591468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 946/1936] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:04 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:04.595432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.602922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672f752-f798-472d-ace7-2c0f1cf83eb2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.607817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.608795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.610281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.610281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] PATH_INFO: `/v3/users/0f9977641a394011bfaa1bb0d52678b4` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.610490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9977641a394011bfaa1bb0d52678b4)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.612597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.621424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8a237-1dc3-48fc-b3ff-d969e390476a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.649842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.650461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.650733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.650999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.651679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.653312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.665240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 947/1937] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:04.668951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.680916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a97c9-9c08-4c4d-b1d3-2b6db280daf9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.686070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.686760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.687041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.687314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] PATH_INFO: `/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90/users/a8986482bd3144f0be40a5741d127864/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.703405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4ec0b6587241b89bf7e82f311ffc90, user_id=a8986482bd3144f0be40a5741d127864, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.706317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.856668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 948/1938] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:04 2022] PUT /identity/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90/users/a8986482bd3144f0be40a5741d127864/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:04.860570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.912746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9977641a394011bfaa1bb0d52678b4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:04.913559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9977641a394011bfaa1bb0d52678b4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:04.932851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9977641a394011bfaa1bb0d52678b4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:04.945058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84980f45-655a-47c8-b74e-1477e1ceba64 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.950050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.950728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.951014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.951284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.951973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.953676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:04.966749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 949/1939] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:04.970988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:04.981242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71e0f-87ee-42f0-8ea3-45594c656814 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:04.986352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:04.987077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:04.987426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:04.987762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] PATH_INFO: `/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90/users/a8986482bd3144f0be40a5741d127864/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:04.989750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4ec0b6587241b89bf7e82f311ffc90, user_id=a8986482bd3144f0be40a5741d127864, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:04.993222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab7849-a83c-453d-93ff-2101a02785e9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:05.003451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 950/1940] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:04 2022] PUT /identity/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90/users/a8986482bd3144f0be40a5741d127864/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:05.017172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.017595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.018105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.046943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] MFA Rules not processed for user `a8986482bd3144f0be40a5741d127864`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:05.048193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.069061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9977641a394011bfaa1bb0d52678b4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:05.069775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-aa52ec2e-2ddd-47cd-9a97-55bc623c6733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9977641a394011bfaa1bb0d52678b4'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:05.094826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 991/1941] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:04 2022] DELETE /identity/v3/users/0f9977641a394011bfaa1bb0d52678b4 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:05.101106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.102144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:05.102806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.103087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.126859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:05.128365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.131148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 951/1942] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:05.144790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:05.146282 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.225633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 992/1943] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:05.227815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c3ed8-40f8-489c-83c6-ee27be3fcb13 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:05.261687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:05.265291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.337534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:05.338275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.338664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.339571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.356255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c81ad-adb1-471d-8891-67ee72270abc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:05.364993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:05.370807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.373120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.373482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.397388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:05.399653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:05.412782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 993/1944] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:05.459131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:05.460705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:05.471776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 952/1945] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:05.788188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:05.798262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900a61a-aea6-4d07-bb40-9dce57c20be9 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:05.801895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:05.802505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.802782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.804333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.805575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.874117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:05.875688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:05.887074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 994/1946] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:05.908136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:05.909575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:05.927473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42731b-ac09-40b5-ae0b-6ae8a99d0658 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:05.955142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:05.955793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:05.956254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:05.956529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] PATH_INFO: `/v3/projects/918892c7ce41473d8dce5aa383727278` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:05.957878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=918892c7ce41473d8dce5aa383727278)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:05.959796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:06.129961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918892c7ce41473d8dce5aa383727278', 'request_id': 'req-41363c96-d195-47c7-afb5-369520334393'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.136877 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918892c7ce41473d8dce5aa383727278', 'request_id': 'req-41363c96-d195-47c7-afb5-369520334393'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.214776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918892c7ce41473d8dce5aa383727278', 'request_id': 'req-41363c96-d195-47c7-afb5-369520334393'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.230817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] The token cache is being invalidate because project 918892c7ce41473d8dce5aa383727278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:06.230817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.231107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.233367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 953/1947] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:05 2022] DELETE /identity/v3/projects/918892c7ce41473d8dce5aa383727278 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:06.237432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:06.250035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764fa2-4333-43f0-89c7-4ad6457e68b7 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:06.321021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:06.321697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:06.322139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:06.322449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] PATH_INFO: `/v3/users/e06cc018f16145baa47ee565cbe1aa5b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:06.323591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06cc018f16145baa47ee565cbe1aa5b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:06.325548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:06.335393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:06.563034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06cc018f16145baa47ee565cbe1aa5b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.564332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06cc018f16145baa47ee565cbe1aa5b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.574686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06cc018f16145baa47ee565cbe1aa5b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.695138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06cc018f16145baa47ee565cbe1aa5b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.696760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-77e42978-973e-499d-a44f-0e43a1731275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06cc018f16145baa47ee565cbe1aa5b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:06.716738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 995/1948] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:06 2022] DELETE /identity/v3/users/e06cc018f16145baa47ee565cbe1aa5b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:07.018607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:07.338153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:07.340251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:07.353973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41363c96-d195-47c7-afb5-369520334393 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:07.417528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:07.418428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:07.418708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:07.418986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] PATH_INFO: `/v3/projects/2eef366e93fc4519b39c11ad66b059ff` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:07.420136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eef366e93fc4519b39c11ad66b059ff)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:07.422125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:07.565850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eef366e93fc4519b39c11ad66b059ff', 'request_id': 'req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:07.572262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eef366e93fc4519b39c11ad66b059ff', 'request_id': 'req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:07.654567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eef366e93fc4519b39c11ad66b059ff', 'request_id': 'req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:07.670197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] The token cache is being invalidate because project 2eef366e93fc4519b39c11ad66b059ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:07.674640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:07.674640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac5fdae8-5323-4dfb-b73d-7704ba5d1661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:07.674640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 954/1949] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:07 2022] DELETE /identity/v3/projects/2eef366e93fc4519b39c11ad66b059ff => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:07.678212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:07.678725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:07.678837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:07.699385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:07.700596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:07.788817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 996/1950] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:07.796601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:07.796966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:07.797320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:07.823385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:07.824739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:07.838186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 955/1951] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:07.843181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:07.855420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedf200-062b-49c1-bd4b-13731cd2e141 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:07.860404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:07.861013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:07.861324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:07.861599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:07.862210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:07.862538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:07.864197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:07.879780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 997/1952] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:07.881051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:07.890757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaff37f-6d34-4a7a-a4ff-fc137f7804f3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:07.895550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:07.896208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:07.896494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:07.896766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:07.897433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:07.900034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.003898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 956/1953] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:07 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:08.005147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:08.016376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee0d13-1f42-4998-a0ae-9a7f7b5418ec admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:08.080621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:08.081338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.081717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.082476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.083240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:08.085214 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.177067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:08.185157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 998/1954] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:08 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:08.191411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:08.202749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b026-b7b9-4f0b-b73d-f10842c5d331 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:08.207736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:08.208272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.208566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.214711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.214711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:08.214711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.224236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 957/1955] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:08.227185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:08.240867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18de9d5-d5ab-469c-9f0b-6e9a73682dc6 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:08.249389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:08.249389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.249389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.249389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.249720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:08.251412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79566b55-82e4-4818-aed0-45963f5e2681 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.263173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 999/1956] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:08.269048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:08.282320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd813f-29f4-4093-b881-1a4262aee8cd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:08.292647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:08.293395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.293834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.294280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] PATH_INFO: `/v3/projects/2ce095bc88e64797a2fa117e66a88a43/users/6852322b0f594743b12c9a2ef797abec/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.312060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce095bc88e64797a2fa117e66a88a43, user_id=6852322b0f594743b12c9a2ef797abec, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:08.314926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.321029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:08.405195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 958/1957] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:08 2022] PUT /identity/v3/projects/2ce095bc88e64797a2fa117e66a88a43/users/6852322b0f594743b12c9a2ef797abec/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:08.416481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.416773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.417072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.450664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] MFA Rules not processed for user `6852322b0f594743b12c9a2ef797abec`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:08.451955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:08.517836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1000/1958] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:08.529634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:08.531390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:08.592941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985f49b-2497-4a3c-a277-a3c20b8b0d06 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:08.596756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:08.597372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:08.597654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:08.597933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:08.607170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:08.608686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87496e6-b8e2-4684-8859-0d169fe290e2 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:08.618672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 959/1959] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:10.107020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:10.366434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:13.635982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:13.965884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:15.011138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:15.012731 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:15.078538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8f40-5799-4ce5-8d17-89c81a5c3dc5 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:15.085192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:15.086720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:15.087111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:15.087504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] PATH_INFO: `/v3/users/c14aeccc28d3415aa7d21686e6a0dd1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:15.088844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14aeccc28d3415aa7d21686e6a0dd1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:15.090941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:15.346650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14aeccc28d3415aa7d21686e6a0dd1a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:15.347535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14aeccc28d3415aa7d21686e6a0dd1a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:15.355664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14aeccc28d3415aa7d21686e6a0dd1a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:15.449171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14aeccc28d3415aa7d21686e6a0dd1a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:15.449734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14aeccc28d3415aa7d21686e6a0dd1a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:15.469830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1001/1960] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:15 2022] DELETE /identity/v3/users/c14aeccc28d3415aa7d21686e6a0dd1a => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:15.474152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:15.474648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:15.474771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:15.493958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:15.495306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:15.514706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:15.556892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 960/1961] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:15.566593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:15.568017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:15.628728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823e10-fb46-4db8-bb0b-2c6712f73649 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:15.633152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:15.633761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:15.634031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:15.634298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:15.643118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:15.644573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:15.654277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1002/1962] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:15.663003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:16.094249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:16.095952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:16.116885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72fd5d-bee1-4083-b05a-d6c70532ca07 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:16.122773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:16.123640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:16.124317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:16.124856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] PATH_INFO: `/v3/projects/df88838ef3784d6883bea9c96c635d05` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:16.126470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df88838ef3784d6883bea9c96c635d05)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:16.129231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:16.232454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:16.233854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:16.250780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777fd3c-b87f-488f-8c3f-147b5bcf6659 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:16.257343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:16.258698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:16.259180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:16.259580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] PATH_INFO: `/v3/users/1ded9368eef941c7b8c6646d71d55e29` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:16.261236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ded9368eef941c7b8c6646d71d55e29)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:16.263467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:16.290607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df88838ef3784d6883bea9c96c635d05', 'request_id': 'req-ae2448df-bfd8-47fd-955d-63284d4550cd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.292362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df88838ef3784d6883bea9c96c635d05', 'request_id': 'req-ae2448df-bfd8-47fd-955d-63284d4550cd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.365743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df88838ef3784d6883bea9c96c635d05', 'request_id': 'req-ae2448df-bfd8-47fd-955d-63284d4550cd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.378728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] The token cache is being invalidate because project df88838ef3784d6883bea9c96c635d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:16.379138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.379851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.381459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 961/1963] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:16 2022] DELETE /identity/v3/projects/df88838ef3784d6883bea9c96c635d05 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:16.389325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:16.390846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:16.464080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2448df-bfd8-47fd-955d-63284d4550cd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:16.468714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:16.469345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:16.469623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:16.469895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] PATH_INFO: `/v3/users/991d1cd8279d4862899a870f43a8f0e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:16.471025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] RBAC: Authorizing `identity:delete_user(user_id=991d1cd8279d4862899a870f43a8f0e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:16.473168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:16.514660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ded9368eef941c7b8c6646d71d55e29'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.516739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ded9368eef941c7b8c6646d71d55e29'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.524808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ded9368eef941c7b8c6646d71d55e29'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.629937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:16.632964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ded9368eef941c7b8c6646d71d55e29'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.634130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4bc21bc7-8553-405a-9ff1-d34c13f97248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ded9368eef941c7b8c6646d71d55e29'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.672720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1003/1964] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:16 2022] DELETE /identity/v3/users/1ded9368eef941c7b8c6646d71d55e29 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:16.674605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:16.674896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:16.675173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:16.706691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:16.714309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:16.772029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1004/1965] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:16.782807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:16.784338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:16.808360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991d1cd8279d4862899a870f43a8f0e6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.809432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991d1cd8279d4862899a870f43a8f0e6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.817300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991d1cd8279d4862899a870f43a8f0e6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.848237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808dbca-3960-4616-9bb3-539dfff7b434 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:16.852019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:16.852672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:16.852954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:16.853224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:16.862425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:16.863932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:16.874423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1005/1966] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:16.908827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991d1cd8279d4862899a870f43a8f0e6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.910712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991d1cd8279d4862899a870f43a8f0e6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:16.947846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 962/1967] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:16 2022] DELETE /identity/v3/users/991d1cd8279d4862899a870f43a8f0e6 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:17.543176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:17.543675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:17.543767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:17.543859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:17.556886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e7cf-1df6-48ac-b185-d8461fc6d8d3 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:17.620708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:17.621346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:17.621625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:17.621896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] PATH_INFO: `/v3/projects/59838991323d4228be05d3b399b54224` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:17.622339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84964638-4a0c-46b8-8121-e1b2e626050f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:17.623040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=59838991323d4228be05d3b399b54224)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:17.625033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:17.627718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:17.628478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:17.628759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:17.629033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] PATH_INFO: `/v3/projects/e1ab8de906764692b7ff89614976486f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:17.630273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ab8de906764692b7ff89614976486f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:17.632580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:17.642063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:17.798396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59838991323d4228be05d3b399b54224', 'request_id': 'req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.804398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59838991323d4228be05d3b399b54224', 'request_id': 'req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.810869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59838991323d4228be05d3b399b54224', 'request_id': 'req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.846323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ab8de906764692b7ff89614976486f', 'request_id': 'req-a7c59538-68b0-4ff9-8c04-76381bd7799e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.853878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ab8de906764692b7ff89614976486f', 'request_id': 'req-a7c59538-68b0-4ff9-8c04-76381bd7799e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.940403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] The token cache is being invalidate because project 59838991323d4228be05d3b399b54224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:17.946166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.946166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4e1ad81f-a6ee-4b72-a5df-9ccc5003b0ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:17.946166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1006/1968] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:17 2022] DELETE /identity/v3/projects/59838991323d4228be05d3b399b54224 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:17.948333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09f42e-877b-429c-a0b0-6d8396717f7c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:17.948618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09f42e-877b-429c-a0b0-6d8396717f7c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:17.948920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09f42e-877b-429c-a0b0-6d8396717f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:17.968816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ea09f42e-877b-429c-a0b0-6d8396717f7c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:17.970047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ea09f42e-877b-429c-a0b0-6d8396717f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:18.030179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1007/1969] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:18.033865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ab8de906764692b7ff89614976486f', 'request_id': 'req-a7c59538-68b0-4ff9-8c04-76381bd7799e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.050298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.050298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.050298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.050526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] The token cache is being invalidate because project e1ab8de906764692b7ff89614976486f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:18.050526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.050526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.054856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 963/1970] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:17 2022] DELETE /identity/v3/projects/e1ab8de906764692b7ff89614976486f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:18.059768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.061416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:18.070721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:18.076470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59538-68b0-4ff9-8c04-76381bd7799e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.081724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:18.081903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.082668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.083051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.083430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] PATH_INFO: `/v3/users/a8986482bd3144f0be40a5741d127864` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.084722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8986482bd3144f0be40a5741d127864)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.089180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.104485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1008/1971] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:18.105551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.120566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0497e45-cb08-4a28-b337-c99ad98e02e1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.125737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.126357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.126642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.127077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.127743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:18.128040 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.129649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.143321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1009/1972] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:18.153567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.159683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea44ca-060a-4b40-a47a-9c89e04dcc6d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.165263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.165898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.166207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.166484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.167117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.169088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.280895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:18.343306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1010/1973] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:18 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:18.344282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.406676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8b4b5-e251-4ca1-b3d3-3ee5ce616a1e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.411315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.411920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.412238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.412517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.413140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.415185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.486317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8986482bd3144f0be40a5741d127864'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.487133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8986482bd3144f0be40a5741d127864'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.494294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8986482bd3144f0be40a5741d127864'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.512173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1011/1974] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:18 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:18.521699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.532164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95acaa-5126-40a1-94a3-d72952f8a902 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.536953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.537564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.537840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.538110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.538856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.540692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.550718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1012/1975] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:18.555270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.559836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8986482bd3144f0be40a5741d127864'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.560737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8986482bd3144f0be40a5741d127864'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:18.565426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea17166-4ad0-4fef-ade8-bf036fff1a82 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.571600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.572265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.572549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.572824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.573764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.575050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95785f1a-0ef8-4356-97f6-d9a2279842ba admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.581984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 964/1976] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:18 2022] DELETE /identity/v3/users/a8986482bd3144f0be40a5741d127864 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:18.586114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1013/1977] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:18.597221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:18.601451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:18.687835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba31f5-d88d-4548-9856-ea9f96611156 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:18.692644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:18.693319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.693605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.693883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] PATH_INFO: `/v3/projects/8e2950230786434db7d680b2ea7224a7/users/2f9cacd4982b426b9ffb949896f83f56/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.709802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2950230786434db7d680b2ea7224a7, user_id=2f9cacd4982b426b9ffb949896f83f56, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:18.712848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:18.852391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 965/1978] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:18 2022] PUT /identity/v3/projects/8e2950230786434db7d680b2ea7224a7/users/2f9cacd4982b426b9ffb949896f83f56/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:18.864986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:18.865269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:18.865538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:18.897720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] MFA Rules not processed for user `2f9cacd4982b426b9ffb949896f83f56`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:18.899217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:18.993894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1014/1979] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:18.998558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.064243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e9afa-185d-4be5-80af-d5af44ff1a38 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.068923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.069529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.070526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.070804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.071426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.073402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.094366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.095792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:19.110307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d624-f815-4535-bc90-8a2f70202941 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.114984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.116319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.117220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.117600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] PATH_INFO: `/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.118814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4ec0b6587241b89bf7e82f311ffc90)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.120895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.207844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 966/1980] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:18 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:19.209241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.275478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4ec0b6587241b89bf7e82f311ffc90', 'request_id': 'req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:19.280856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4ec0b6587241b89bf7e82f311ffc90', 'request_id': 'req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:19.286951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4ec0b6587241b89bf7e82f311ffc90', 'request_id': 'req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:19.327182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af935f0d-8bbd-4053-a74b-69bb3b068c7f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.382098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.383380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.383723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.383962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.384627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.390320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.397189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] The token cache is being invalidate because project 5a4ec0b6587241b89bf7e82f311ffc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:19.397612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:19.398294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4d8e751e-69cb-40ff-a1e8-d0cebf5a6ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:19.400127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1015/1981] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:19 2022] DELETE /identity/v3/projects/5a4ec0b6587241b89bf7e82f311ffc90 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:19.404977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acf0f-fc66-45cc-8d28-333f7b0d644d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.405266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acf0f-fc66-45cc-8d28-333f7b0d644d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.405545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acf0f-fc66-45cc-8d28-333f7b0d644d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.424784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6f7acf0f-fc66-45cc-8d28-333f7b0d644d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:19.426137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6f7acf0f-fc66-45cc-8d28-333f7b0d644d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:19.434628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1016/1982] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:19.442263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.442551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.442830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.462484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:19.463697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:19.471363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1017/1983] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:19.475391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.484943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d063bd-f919-4f8f-97a5-849b5772fccb None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.490136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.499473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.509838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1018/1984] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:07:19.510796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.520378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6e8dc-fb50-4c97-a928-b73d54c9d3ee admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.525052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.526068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.526068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.526396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.526972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.528950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.534635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 967/1985] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/users => generated 297 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:19.539855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.542942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:19.555983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79399692-bc15-4f22-ba26-075c669f4d91 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.561175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.561801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.562205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.562547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.563299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.565090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.576253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 968/1986] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:19.580548 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.590163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d711cc-2e69-4e9e-b09e-78e5a9f9474d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.594775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.595401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.595680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.595951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] PATH_INFO: `/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89/users/4c507b312d4b4119b6da080da0adfe4d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.611381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd462a6f9e14b47a1a38bb30b689d89, user_id=4c507b312d4b4119b6da080da0adfe4d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.614075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.665827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1019/1987] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:19.670124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.715200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 969/1988] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:19 2022] PUT /identity/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89/users/4c507b312d4b4119b6da080da0adfe4d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:19.720960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.744031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ea1ec-8e50-4f2f-8546-223d46651813 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.787707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc488859-876d-4bc8-a0e0-e85a6724c508 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.793320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.794052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.794337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.794668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.795324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.797001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.807585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 970/1989] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:19.811762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.811904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.812662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.813181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.813468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.814102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.816276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.822665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562594a7-9be7-4960-b28f-c5eac22ddc8e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.827338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:19.827990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.828388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.828669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] PATH_INFO: `/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89/users/4c507b312d4b4119b6da080da0adfe4d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.830578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd462a6f9e14b47a1a38bb30b689d89, user_id=4c507b312d4b4119b6da080da0adfe4d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:19.833305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699d9f-8460-431c-975b-05fe9ea50ab6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:19.851140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 971/1990] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:19 2022] PUT /identity/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89/users/4c507b312d4b4119b6da080da0adfe4d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:19.860953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:19.861533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:19.861711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:19.896436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] MFA Rules not processed for user `4c507b312d4b4119b6da080da0adfe4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:19.896436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:19.937447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1020/1991] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/users => generated 287 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:19.938424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.949810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22883f-5c21-47ef-87b5-9df75a4218c0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:19.969137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 972/1992] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:19.981241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:19.982723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:20.012657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.013371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.013655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.013933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.014605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.016347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.027082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1021/1993] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:20.030989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:20.040706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5c2c-b909-4ca7-b569-9a6d864aa712 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:20.045609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.046283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.046598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.046873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.051113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.051217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbb2e5-1a0a-498a-8895-ee014dd27eda None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:20.051313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.052974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.053579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.053859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.054738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.064392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1022/1994] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:20.065535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:20.066111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.067613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4441287-dc88-431d-874f-f0c4b7be838f service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.080253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd6da1-739f-499b-8bfc-4ec241c3caf3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:20.080344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 973/1995] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:20.084351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.085106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.085493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.085925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] PATH_INFO: `/v3/projects/d1f677a8a80e4591ab642d64f4008c00/users/1f2a411b051147e58a94dfa03413539b/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.103146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f677a8a80e4591ab642d64f4008c00, user_id=1f2a411b051147e58a94dfa03413539b, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.106061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.209855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1023/1996] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:20 2022] PUT /identity/v3/projects/d1f677a8a80e4591ab642d64f4008c00/users/1f2a411b051147e58a94dfa03413539b/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:20.215143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.215676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.215962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.243986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:20.245238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:20.307970 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:20.309838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 974/1997] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:20.320729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:20.322593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:20.349289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:20.351734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:20.391768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0abe-612b-4dde-a34e-3317d718dfcc admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:20.395479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.396104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.396410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.396672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.407896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.409405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5455377-cd00-4e89-a7e7-16e862291ddb service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.415022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d261b42-5f31-4ac2-88ae-ead616a59745 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:20.418813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:20.419459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:20.419705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:20.419977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:20.421001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1024/1998] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:20.422442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:20.482169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:20.483653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:20.493344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 975/1999] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:20.997571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:21.886741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de430-416b-4f80-8aa2-c76fac6daca3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:21.887052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de430-416b-4f80-8aa2-c76fac6daca3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:21.887509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de430-416b-4f80-8aa2-c76fac6daca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:21.919006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-830de430-416b-4f80-8aa2-c76fac6daca3 None None] MFA Rules not processed for user `1f2a411b051147e58a94dfa03413539b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:21.921487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-830de430-416b-4f80-8aa2-c76fac6daca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:21.998052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1025/2000] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:22.012746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:22.021187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b9f89-61be-4c5c-9a3d-d99c4f8da5b5 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:22.025585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:22.026187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:22.026494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:22.026740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:22.034851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:22.036308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:22.045026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 976/2001] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:23.013085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:23.680818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:24.972327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:24.972976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:24.972976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:24.992639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] MFA Rules not processed for user `1f2a411b051147e58a94dfa03413539b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:24.993936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:25.002131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1026/2002] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:25.930086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:26.632883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:27.180790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:27.372357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:28.126039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:28.193253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:28.721076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:28.732129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:28.746170 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451a804-0146-4c01-97c6-0ae27be12bfb service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:28.750718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:28.751353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:28.751634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:28.751907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] PATH_INFO: `/v3/users/4529d06c4eda48f788ce7c0327366377` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:28.753095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4529d06c4eda48f788ce7c0327366377)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:28.755039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:29.014507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4529d06c4eda48f788ce7c0327366377'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.015248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4529d06c4eda48f788ce7c0327366377'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.024874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4529d06c4eda48f788ce7c0327366377'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.148695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4529d06c4eda48f788ce7c0327366377'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.149387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-bc5d918e-d3a2-4e44-8a76-889145575acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4529d06c4eda48f788ce7c0327366377'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.169585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 977/2003] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:28 2022] DELETE /identity/v3/users/4529d06c4eda48f788ce7c0327366377 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:29.257740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:29.455643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:29.468612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e109963-9b1e-4fe7-80b5-90fddd3b3ea7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:29.529453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:29.530011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.530287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.530557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] PATH_INFO: `/v3/projects/eeb0f7283fb84ea48fd468f22c721b8b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.531762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb0f7283fb84ea48fd468f22c721b8b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:29.533757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:29.636257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb0f7283fb84ea48fd468f22c721b8b', 'request_id': 'req-fd678282-a202-4f64-a6df-53f84b513c39'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.641616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb0f7283fb84ea48fd468f22c721b8b', 'request_id': 'req-fd678282-a202-4f64-a6df-53f84b513c39'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.648621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb0f7283fb84ea48fd468f22c721b8b', 'request_id': 'req-fd678282-a202-4f64-a6df-53f84b513c39'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.740201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] The token cache is being invalidate because project eeb0f7283fb84ea48fd468f22c721b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:29.740501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.741293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:29.746380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1027/2004] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:29 2022] DELETE /identity/v3/projects/eeb0f7283fb84ea48fd468f22c721b8b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:29.749786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa18ea-acd1-433c-a523-a0b4a546b963 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.750671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa18ea-acd1-433c-a523-a0b4a546b963 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.751020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa18ea-acd1-433c-a523-a0b4a546b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.761707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:29.763149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:29.772029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-dcaa18ea-acd1-433c-a523-a0b4a546b963 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:29.774588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-dcaa18ea-acd1-433c-a523-a0b4a546b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:29.840062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd678282-a202-4f64-a6df-53f84b513c39 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:29.841510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 978/2005] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:29.849360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.849735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.850086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.853005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:29.858387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.858387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.858387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] PATH_INFO: `/v3/users/3cc219984a5942b1bd39ad2265bbe85e` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.858387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc219984a5942b1bd39ad2265bbe85e)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:29.858387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:29.871564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:29.872802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:29.882763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 979/2006] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:29.887400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:29.896553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2c18e-b138-4a98-918c-cafa40496743 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:29.901938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:29.902526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.902804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.903080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.903722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:29.904286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:29.906102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:29.918700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 980/2007] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:07:29.924623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:29.937770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a1921-d8ca-468c-9f86-7877115735b2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:29.943322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:29.943976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:29.944342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:29.944740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:29.950275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:29.950275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.064337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 981/2008] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:29 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:30.067472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.147457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c71a-0253-4ab6-bafc-1c65f791f525 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.152944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.153815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.154103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.154380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.155013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.156969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.164354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:30.198297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc219984a5942b1bd39ad2265bbe85e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:30.198442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc219984a5942b1bd39ad2265bbe85e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:30.212579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc219984a5942b1bd39ad2265bbe85e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:30.270596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 982/2009] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:30 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:30.275415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.285154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc219984a5942b1bd39ad2265bbe85e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:30.285711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2969991-595b-4dee-9383-037e49b7135e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.286479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b27e41f6-71ca-44a5-9723-19083cf92b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc219984a5942b1bd39ad2265bbe85e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:30.290958 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.291677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.292038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.292414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.293178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.294902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.308758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1028/2010] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:29 2022] DELETE /identity/v3/users/3cc219984a5942b1bd39ad2265bbe85e => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:30.311670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.312025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.312412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.327907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 983/2011] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:30.327907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.329727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:30.334862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:30.337168 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:30.343956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca2451-5145-4ad3-9648-c3be5d66d250 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.412372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1029/2012] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:30.418331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.421345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:30.435143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.435779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.436109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.436482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.437119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.438998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.450154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 984/2013] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:30.459682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.469474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1eda4-8047-47d6-a923-79c9c4184077 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.474180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.474914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.475228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.475501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] PATH_INFO: `/v3/projects/24e83f69f6834d98b850b6446f5760a0/users/9945766921aa412a88e9f41691d15611/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.485328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4631b8-2362-4217-aad1-878e0834687a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.490512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.492482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.492482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.492482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.495458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e83f69f6834d98b850b6446f5760a0, user_id=9945766921aa412a88e9f41691d15611, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.498312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.502349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.503805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4efb6-6b36-4521-bfd3-ca0f1f57ce80 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.514948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1030/2014] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:30.633073 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 985/2015] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:30 2022] PUT /identity/v3/projects/24e83f69f6834d98b850b6446f5760a0/users/9945766921aa412a88e9f41691d15611/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:30.643130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.643448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.643839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.671785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] MFA Rules not processed for user `9945766921aa412a88e9f41691d15611`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:30.673081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:30.743134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1031/2016] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:30.747833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:30.831907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9dc3-b422-46c9-9adc-0d5300b71731 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:30.838041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:30.838664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:30.838972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:30.839247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:30.839885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:30.841865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:30.988413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 986/2017] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:30 2022] POST /identity/v3/projects => generated 403 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:30.993464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.004190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e73512-1424-47fa-b4ad-98246a28ed90 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.028118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.029783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.114811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.116226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.116669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.116943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.117653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.120235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.143296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f75660-06e1-43c2-ae5f-a20bb245526d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.152604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.153342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.153694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.154067 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] PATH_INFO: `/v3/projects/31b060632a774671af1631f203c763c2` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.155972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b060632a774671af1631f203c763c2)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.159606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.283319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1032/2018] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:30 2022] POST /identity/v3/users => generated 308 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:31.284371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.294608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388cdc-960d-4436-8a29-788986fc2740 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.299281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.299874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.300241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.300732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.301408 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.303106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.315206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1033/2019] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:31.321574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.340003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b060632a774671af1631f203c763c2', 'request_id': 'req-c75f41b1-faf7-4be4-b2ec-b599386d2ede'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:31.341105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fcfa0-1706-4889-9633-fa291e57916d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.345922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b060632a774671af1631f203c763c2', 'request_id': 'req-c75f41b1-faf7-4be4-b2ec-b599386d2ede'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:31.410276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.410941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.411220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.411490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] PATH_INFO: `/v3/projects/d2b0a677935b402893b17fd895126b53/users/1166402ceccf475d9b31ffd5d2035028/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.426770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b0a677935b402893b17fd895126b53, user_id=1166402ceccf475d9b31ffd5d2035028, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.430965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.519527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b060632a774671af1631f203c763c2', 'request_id': 'req-c75f41b1-faf7-4be4-b2ec-b599386d2ede'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:31.540374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] The token cache is being invalidate because project 31b060632a774671af1631f203c763c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:31.541832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:31.542526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c75f41b1-faf7-4be4-b2ec-b599386d2ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:31.544781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 987/2020] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:31 2022] DELETE /identity/v3/projects/31b060632a774671af1631f203c763c2 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:31.549030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb81bfc-ec19-4916-b87b-a63e26bbac3a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.549390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb81bfc-ec19-4916-b87b-a63e26bbac3a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.549738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb81bfc-ec19-4916-b87b-a63e26bbac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.579653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-3cb81bfc-ec19-4916-b87b-a63e26bbac3a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:31.579653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3cb81bfc-ec19-4916-b87b-a63e26bbac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.597309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 988/2021] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:31.606501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.606789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.608325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.608419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1034/2022] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:31 2022] PUT /identity/v3/projects/d2b0a677935b402893b17fd895126b53/users/1166402ceccf475d9b31ffd5d2035028/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:31.611031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.612653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.627810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:31.629182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.680552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63108-d196-4957-8ee1-42501f25cf80 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.685342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.685954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.686232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.686505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.687149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.688776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.696451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 989/2023] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:31.699751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1035/2024] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:31.700950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.703505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.711008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba1c63-1b52-41c1-844d-1a046c5aefd8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.713484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3524df-254c-466e-a389-2d9041e88342 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.716539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.717525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.718012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.718363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.718619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.719012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.719293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.719566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] PATH_INFO: `/v3/projects/d2b0a677935b402893b17fd895126b53/users/1166402ceccf475d9b31ffd5d2035028/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.720202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:31.720771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.721813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b0a677935b402893b17fd895126b53, user_id=1166402ceccf475d9b31ffd5d2035028, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.724640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.724836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988e143-b9b3-4380-8cd2-aa94c58137f4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.737291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1036/2025] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:31 2022] PUT /identity/v3/projects/d2b0a677935b402893b17fd895126b53/users/1166402ceccf475d9b31ffd5d2035028/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:31.748344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 990/2026] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:31.748344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.750619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.750960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.751243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.755391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132406-6463-4245-83cc-c36d58799724 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.791166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] MFA Rules not processed for user `1166402ceccf475d9b31ffd5d2035028`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:31.793599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.814614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:31.821411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.822129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.822491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.822865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.823581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.825863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.871269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1037/2027] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:31.882346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:31.883908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:31.940602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a21e2-f26f-4092-94f3-a85aed1c7b0b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:31.944214 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:31.944829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:31.945105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:31.945379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:31.954458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:31.955975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:31.961756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 991/2028] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:31 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:31.973222 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1038/2029] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:31 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:31.977032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.064785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4dc83-5418-44c5-9fba-44d61b036198 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.070043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.073468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.073468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.073468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.073468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.074293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.131274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.146141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9de5e-ea10-4fda-8566-0cdd3a6cf91f service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.151177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.151836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.152117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.152533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] PATH_INFO: `/v3/users/9945766921aa412a88e9f41691d15611` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.153753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9945766921aa412a88e9f41691d15611)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.155849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.173687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 992/2030] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:31 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:32.183245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.189834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917f1ec-2b45-4ea9-91b2-79d4b353a5ca admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.195166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.195901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.196272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.196632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.197344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.199019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.208862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 993/2031] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:32.213566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.223902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec7fb1-0a95-43a5-92ff-671a87f2af47 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.229810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.230426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.230711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.230992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.231643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.233336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.240693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:32.247242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 994/2032] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:32.251363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.261844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872e2a-619b-427d-8757-2d867b6c0e17 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.268173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.268863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.269151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.269565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] PATH_INFO: `/v3/projects/f122aa8ceea64d2f8f57230a8a00b0d4/users/4ee8348c14094a7187efd6dd355e3f5f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.288641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f122aa8ceea64d2f8f57230a8a00b0d4, user_id=4ee8348c14094a7187efd6dd355e3f5f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.291771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535978-9bef-4989-a13e-8843ba9b2a95 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.439075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 995/2033] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:32 2022] PUT /identity/v3/projects/f122aa8ceea64d2f8f57230a8a00b0d4/users/4ee8348c14094a7187efd6dd355e3f5f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:32.457850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.458218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.458636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.488765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9945766921aa412a88e9f41691d15611'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:32.489806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9945766921aa412a88e9f41691d15611'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:32.493744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] MFA Rules not processed for user `4ee8348c14094a7187efd6dd355e3f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:32.494979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:32.497307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9945766921aa412a88e9f41691d15611'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:32.562693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 996/2034] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:32.569946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9945766921aa412a88e9f41691d15611'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:32.570880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-06118df0-a618-4521-a44f-64a7143099ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9945766921aa412a88e9f41691d15611'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:32.575722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.596010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1039/2035] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:32 2022] DELETE /identity/v3/users/9945766921aa412a88e9f41691d15611 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:32.600351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.600846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.601528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.626604 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:32.628303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:32.658551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca3ce2-39e8-4db5-8542-e94ff48a833d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.711286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1040/2036] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:32.723377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:32.725885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:32.728519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.733464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.733945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.734427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.737577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:32.839656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59a6bd-ee23-4c9d-b41f-695c06e140f0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:32.847149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:32.847149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:32.847149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:32.848014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:32.860492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.862255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.868611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:32.870751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:32.874479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1041/2037] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:32.880878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 997/2038] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:33.379499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:33.380950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:33.394905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb783-40c7-4bd1-bacd-b4f0fd58ea12 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:33.399882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:33.400577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:33.400886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:33.401183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] PATH_INFO: `/v3/projects/24e83f69f6834d98b850b6446f5760a0` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:33.407539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e83f69f6834d98b850b6446f5760a0)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:33.407539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:33.559693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e83f69f6834d98b850b6446f5760a0', 'request_id': 'req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:33.567487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e83f69f6834d98b850b6446f5760a0', 'request_id': 'req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:33.584253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e83f69f6834d98b850b6446f5760a0', 'request_id': 'req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:33.712314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] The token cache is being invalidate because project 24e83f69f6834d98b850b6446f5760a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:33.712694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:33.713358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:33.714845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1042/2039] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:33 2022] DELETE /identity/v3/projects/24e83f69f6834d98b850b6446f5760a0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:33.718786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:33.730747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d2bb-b0aa-4d1f-a1d9-b630c93d3e22 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:33.793773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:33.806197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:33.806197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:33.806197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] PATH_INFO: `/v3/users/1166402ceccf475d9b31ffd5d2035028` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:33.806197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1166402ceccf475d9b31ffd5d2035028)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:33.806197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:34.042587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1166402ceccf475d9b31ffd5d2035028'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.044103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1166402ceccf475d9b31ffd5d2035028'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.051233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1166402ceccf475d9b31ffd5d2035028'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.128477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1166402ceccf475d9b31ffd5d2035028'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.129113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1166402ceccf475d9b31ffd5d2035028'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.147926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 998/2040] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:33 2022] DELETE /identity/v3/users/1166402ceccf475d9b31ffd5d2035028 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:34.215776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:34.217365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.231677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03009d-8c58-4b1f-8f21-77537c5f3c2f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:34.291510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:34.292165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.292452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.292728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] PATH_INFO: `/v3/users/2f9cacd4982b426b9ffb949896f83f56` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.293869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9cacd4982b426b9ffb949896f83f56)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:34.295875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:34.508833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:34.510653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9cacd4982b426b9ffb949896f83f56'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.511336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9cacd4982b426b9ffb949896f83f56'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.518450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9cacd4982b426b9ffb949896f83f56'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.599139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:34.599139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.608778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da660c-8c6d-40a1-ba2b-bff4dcb3db47 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:34.613966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:34.614634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.614944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.615509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] PATH_INFO: `/v3/projects/d2b0a677935b402893b17fd895126b53` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.617354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b0a677935b402893b17fd895126b53)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:34.617790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9cacd4982b426b9ffb949896f83f56'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.619065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16c57486-4eb5-47ad-83bc-bf3972638205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9cacd4982b426b9ffb949896f83f56'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.619308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:34.657293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1043/2041] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:34 2022] DELETE /identity/v3/users/2f9cacd4982b426b9ffb949896f83f56 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:34.664118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.664584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.664850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.684193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:34.685341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.691225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:34.750553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1044/2042] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:34.752422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b0a677935b402893b17fd895126b53', 'request_id': 'req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.761212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:34.769350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b0a677935b402893b17fd895126b53', 'request_id': 'req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.769490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.831559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec6a0-5264-4f62-a2bf-fae63c7867f3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:34.884936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b0a677935b402893b17fd895126b53', 'request_id': 'req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.885197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:34.885837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.886112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.886392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.898031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] The token cache is being invalidate because project d2b0a677935b402893b17fd895126b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:34.898827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.899622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-be25fbc2-1c86-4da8-b9b2-3f4a59668f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:34.901399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 999/2043] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:34 2022] DELETE /identity/v3/projects/d2b0a677935b402893b17fd895126b53 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:34.908184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730c31d-6bd0-4145-9df5-3ae99d9797e4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.908184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730c31d-6bd0-4145-9df5-3ae99d9797e4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.908184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730c31d-6bd0-4145-9df5-3ae99d9797e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.942589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7730c31d-6bd0-4145-9df5-3ae99d9797e4 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:34.943866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7730c31d-6bd0-4145-9df5-3ae99d9797e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.951293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:34.961345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:34.961508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1000/2044] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:34.962216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:34.962508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:34.962788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:34.964271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1045/2045] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:34.983334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:34.984592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:34.993366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1001/2046] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:34.997696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:34.999344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:35.015925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2be46-bcfe-4471-a1fe-52b9548cdc9f service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.021539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.022302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.022924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.025008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.025008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:35.025008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.026884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.039932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1046/2047] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:35.044351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.056520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab3b6b-289c-4819-bf6c-3564d96b0110 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.061953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.062668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.063202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.063571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.064331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.066355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.222034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1002/2048] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:35 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:35.235005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.248552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fff53e-f9d6-4d10-b4e2-0f7d03516405 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.315204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.316031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.316356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.316701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.317567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.319659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.470949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1047/2049] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:35 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:35.471989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.472452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.474399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:35.484994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16667fba-922a-42a4-ba23-3f0f6fa4d357 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.487482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224f2e7-1fd7-4a47-91f0-8d76184e20d5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.491887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.493239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.493525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.493815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.494285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.494453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.495039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.495398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.495844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] PATH_INFO: `/v3/projects/8e2950230786434db7d680b2ea7224a7` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.496293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.497363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2950230786434db7d680b2ea7224a7)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.499651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.509893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1048/2050] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:35.513784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.524530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb728d7-fe86-4ca1-ae71-0cb3e9a862a3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.531698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.532343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.532648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.532917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.533707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.535283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.545886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1049/2051] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:07:35.549814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.559495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd2f1b-fce9-4c82-b12b-ba84c4b3d558 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.564080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.564738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.565014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.565283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] PATH_INFO: `/v3/projects/6e5dcb22055547d29deaf8c50f377778/users/8ebed906710b4faeba23cf7caf973138/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.582267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5dcb22055547d29deaf8c50f377778, user_id=8ebed906710b4faeba23cf7caf973138, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.583013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dfd7e-aa90-4aaf-b995-f51908d0e995 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.662054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1050/2052] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:35 2022] PUT /identity/v3/projects/6e5dcb22055547d29deaf8c50f377778/users/8ebed906710b4faeba23cf7caf973138/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:35.672824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.673105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.673380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.680716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2950230786434db7d680b2ea7224a7', 'request_id': 'req-21dfa038-ffa8-4f2b-82ba-495ba80425e7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:35.686420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2950230786434db7d680b2ea7224a7', 'request_id': 'req-21dfa038-ffa8-4f2b-82ba-495ba80425e7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:35.702280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] MFA Rules not processed for user `8ebed906710b4faeba23cf7caf973138`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:35.703504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:35.748214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2950230786434db7d680b2ea7224a7', 'request_id': 'req-21dfa038-ffa8-4f2b-82ba-495ba80425e7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:35.762071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] The token cache is being invalidate because project 8e2950230786434db7d680b2ea7224a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:35.763367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:35.763367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:35.765279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1003/2053] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:35 2022] DELETE /identity/v3/projects/8e2950230786434db7d680b2ea7224a7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:35.773588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.775081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:35.784433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1051/2054] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:35.785629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:35.842390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfa038-ffa8-4f2b-82ba-495ba80425e7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.850070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.850949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.851226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.851527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] PATH_INFO: `/v3/users/4c507b312d4b4119b6da080da0adfe4d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.852700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c507b312d4b4119b6da080da0adfe4d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.853359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51eab-455c-49c3-bfee-bb58471b5a8d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:35.854638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:35.859362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:35.859974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:35.860292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:35.860572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:35.861209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:35.863279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.045704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1052/2055] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:35 2022] POST /identity/v3/projects => generated 377 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:36.046695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.048340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.076917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c507b312d4b4119b6da080da0adfe4d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:36.077600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c507b312d4b4119b6da080da0adfe4d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:36.086160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c507b312d4b4119b6da080da0adfe4d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:36.117558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8bbe-8c31-44d3-a2bb-91c9fd7d12fa admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.122196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.122800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.123079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.123384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.124047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.125990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.210847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c507b312d4b4119b6da080da0adfe4d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:36.211432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c507b312d4b4119b6da080da0adfe4d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:36.238454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1004/2056] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:35 2022] DELETE /identity/v3/users/4c507b312d4b4119b6da080da0adfe4d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:36.261972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1053/2057] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:36 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:36.277256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.278640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.349967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae44c2-053e-43c6-a1c3-a72eca0d62d4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.355902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.357063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.360385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.360385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.360385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.360385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.374429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1005/2058] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:36.375522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.386337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99dab-af4b-4e7e-bb61-f8888a5e34f8 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.391115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.391832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.392109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.394579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] PATH_INFO: `/v3/projects/aaa6621ccd034c48983cc72635c4c387/users/b944a42b78a24159a05cef0589527ed8/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.408894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa6621ccd034c48983cc72635c4c387, user_id=b944a42b78a24159a05cef0589527ed8, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.411886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.518104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1054/2059] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:36 2022] PUT /identity/v3/projects/aaa6621ccd034c48983cc72635c4c387/users/b944a42b78a24159a05cef0589527ed8/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:36.522645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.533766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d40b1d-9cb0-4274-80a6-e69ac41da428 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.594024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.594801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.595090 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.595363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.596025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.597782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76660b17-b1b9-4e84-97ef-c3529d51e474 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.609289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1006/2060] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:36.613462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.624932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2d4bc-4f69-4e72-ba2b-92db0fe78583 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.636467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.636467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.636467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.636467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] PATH_INFO: `/v3/projects/aaa6621ccd034c48983cc72635c4c387/users/b944a42b78a24159a05cef0589527ed8/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.636894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa6621ccd034c48983cc72635c4c387, user_id=b944a42b78a24159a05cef0589527ed8, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.638909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.652206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1055/2061] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:36 2022] PUT /identity/v3/projects/aaa6621ccd034c48983cc72635c4c387/users/b944a42b78a24159a05cef0589527ed8/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:36.663885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.667426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.667426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.696092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] MFA Rules not processed for user `b944a42b78a24159a05cef0589527ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:36.697435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.719722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:36.756176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.759035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.775085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1007/2062] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:36.786762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:36.788982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.842942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5c1de-63cc-4fce-9813-b26c66dc9204 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.848136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.848855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.849137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.849411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] PATH_INFO: `/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.851245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd462a6f9e14b47a1a38bb30b689d89)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.853339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.860338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9eb1d4-606e-4eec-bcb5-c93bfdb45167 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:36.863930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:36.864591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:36.864873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:36.865149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:36.867926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:36.933710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:36.935379 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0ea47-1051-43ff-be11-40f3dcbb30ab service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:36.945347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1008/2063] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:37.060491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:37.061082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd462a6f9e14b47a1a38bb30b689d89', 'request_id': 'req-ccd5857c-faa3-49bc-a1d3-02a121b681d2'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:37.067141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd462a6f9e14b47a1a38bb30b689d89', 'request_id': 'req-ccd5857c-faa3-49bc-a1d3-02a121b681d2'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:37.076240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd462a6f9e14b47a1a38bb30b689d89', 'request_id': 'req-ccd5857c-faa3-49bc-a1d3-02a121b681d2'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:37.229180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] The token cache is being invalidate because project 5bd462a6f9e14b47a1a38bb30b689d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:37.229548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:37.230146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ccd5857c-faa3-49bc-a1d3-02a121b681d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:37.232304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1056/2064] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:36 2022] DELETE /identity/v3/projects/5bd462a6f9e14b47a1a38bb30b689d89 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:37.237075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.237355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.237646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.263535 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:37.264850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:37.325651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1009/2065] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:37.333503 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.333796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.334074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.354367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:37.355592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:37.364310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1057/2066] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:37.372289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.382861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fe6fc-dcf7-491c-bd9a-eb3fa953b7d6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.387639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.388317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.388598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.388871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.389525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:37.389831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.391436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.403443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1010/2067] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:37.407686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.417022 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dee4a-a9c2-48e0-a040-0d91b797833d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.421884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.422522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.422799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.423076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.423738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.425739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.555416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1058/2068] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:37 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:37.559716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.573477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e65d-9721-418a-8d45-9c9ae2272f18 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.634210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.634817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.635092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.635364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.636004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.638063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.718247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1011/2069] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:37 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:37.722486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.732844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fd7b3-609d-43ca-a8a9-85658258638e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.737520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.738149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.738423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.738712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.739344 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.741017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.752163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1059/2070] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:37.760951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.771626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfcde5-a492-4dc0-86f4-258ddbed9c2d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.777502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.778191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.778547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.778884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.779629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.781244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe2a74-50c6-472a-bb85-ee95b94a0b76 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.792206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1012/2071] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:37.796514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:37.807056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda985-c767-4291-b9a7-31c50b98de42 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:37.811932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:37.812704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.814878 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.814878 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] PATH_INFO: `/v3/projects/b43137fa86ef498399df67dda2316ecf/users/ed2d50a2923841af94e493fdfdf50a6d/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:37.833418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43137fa86ef498399df67dda2316ecf, user_id=ed2d50a2923841af94e493fdfdf50a6d, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:37.836725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:37.977215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1060/2072] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:37 2022] PUT /identity/v3/projects/b43137fa86ef498399df67dda2316ecf/users/ed2d50a2923841af94e493fdfdf50a6d/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:37.988650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:37.988966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:37.989311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.031705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] MFA Rules not processed for user `ed2d50a2923841af94e493fdfdf50a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:38.033635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:38.099760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1013/2073] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:38.104385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.166884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86aa988-bcf7-4571-903a-e64b8e4ff54b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.171591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.172231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.172516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.172793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.173432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.175363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.317383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1061/2074] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:38 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:38.319996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.330622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf6a88-342e-4bfd-870d-975f58ffc8b8 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.391077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.391738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.391999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.392308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.392937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.394903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.492343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1014/2075] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:38 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:38.497810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.508622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108aa0ed-9705-4b03-832b-217d3d28d4c9 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.514163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.514764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.515065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.515335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.516002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.517661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.528427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1062/2076] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:38.532724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.543383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423669-7b98-4e82-9ccf-cac22dd8869b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.547996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.548691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.548976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.549248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] PATH_INFO: `/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c/users/e27cc2423d2240ae97a9f890ef55b2b8/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.564457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce77caef3b54ea0a551be9f2c89ae3c, user_id=e27cc2423d2240ae97a9f890ef55b2b8, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.567146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.656634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1015/2077] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:38 2022] PUT /identity/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c/users/e27cc2423d2240ae97a9f890ef55b2b8/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:38.660589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.671656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a2be3-d96f-4116-893e-91021b4ddd77 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.731277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.732051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.732218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.732498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.733152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.734757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778a68-3a74-4cb3-aac3-49ca3fc2158b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.745220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1063/2078] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:38.749649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.762249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96821-de4f-4518-8a8e-86ad75a9dec7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.766992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.767704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.767982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.768289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] PATH_INFO: `/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c/users/e27cc2423d2240ae97a9f890ef55b2b8/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.770217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce77caef3b54ea0a551be9f2c89ae3c, user_id=e27cc2423d2240ae97a9f890ef55b2b8, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:38.773423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:38.794735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1016/2079] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:38 2022] PUT /identity/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c/users/e27cc2423d2240ae97a9f890ef55b2b8/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:38.796668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.796966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.797274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:38.829879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] MFA Rules not processed for user `e27cc2423d2240ae97a9f890ef55b2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:38.831135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:38.917459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1064/2080] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:38.929695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:38.931672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:38.990825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbb0c6-9a51-4008-8b68-eb1b4ae94e9b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:38.994960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:38.995579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:38.995863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:38.996140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:39.005666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:39.008277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b767b8-9add-4311-875b-4779a4648ae1 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:39.018016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1017/2081] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:39.228311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:40.748843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:40.842151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095e1d-da67-4b77-b08d-39f8b4a68341 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:40.847392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:40.848088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:40.848399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:40.848671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] PATH_INFO: `/v3/users/ed2d50a2923841af94e493fdfdf50a6d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:40.849787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2d50a2923841af94e493fdfdf50a6d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:40.860349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:41.140401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2d50a2923841af94e493fdfdf50a6d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:41.141260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2d50a2923841af94e493fdfdf50a6d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:41.153497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2d50a2923841af94e493fdfdf50a6d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:41.216448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:41.260309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2d50a2923841af94e493fdfdf50a6d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:41.261019 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2d50a2923841af94e493fdfdf50a6d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:41.280420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1065/2082] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:40 2022] DELETE /identity/v3/users/ed2d50a2923841af94e493fdfdf50a6d => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:41.284979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:41.285272 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:41.285553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:41.308875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:41.308875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:41.370005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1018/2083] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:41.379825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:41.382242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:41.445350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc6a0e-6095-4177-9d14-2d11cfa0a537 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:41.449041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:41.449776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:41.450053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:41.450320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:41.460276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:41.461826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:41.471745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1066/2084] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:41.970218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:41.971908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:41.984709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e6e55-0255-496a-8900-32596382eeeb None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:41.990300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:41.990963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:41.992121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:41.992470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] PATH_INFO: `/v3/projects/b43137fa86ef498399df67dda2316ecf` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:41.993689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43137fa86ef498399df67dda2316ecf)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:41.995826 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:42.131852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43137fa86ef498399df67dda2316ecf', 'request_id': 'req-cb4be894-a64b-4b2a-b7fa-c1efa652046c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.137166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43137fa86ef498399df67dda2316ecf', 'request_id': 'req-cb4be894-a64b-4b2a-b7fa-c1efa652046c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.274334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43137fa86ef498399df67dda2316ecf', 'request_id': 'req-cb4be894-a64b-4b2a-b7fa-c1efa652046c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.286574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] The token cache is being invalidate because project b43137fa86ef498399df67dda2316ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:42.287023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.287782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.290382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1019/2085] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:41 2022] DELETE /identity/v3/projects/b43137fa86ef498399df67dda2316ecf => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:42.294143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:42.304366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed8ff-287c-4db3-9bf5-e9235ae13c6d service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:42.362685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:42.368339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:42.368339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:42.368339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] PATH_INFO: `/v3/users/e27cc2423d2240ae97a9f890ef55b2b8` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:42.368339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27cc2423d2240ae97a9f890ef55b2b8)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:42.368339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:42.560701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:42.563241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27cc2423d2240ae97a9f890ef55b2b8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.563896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27cc2423d2240ae97a9f890ef55b2b8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.571784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27cc2423d2240ae97a9f890ef55b2b8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.665846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27cc2423d2240ae97a9f890ef55b2b8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.666411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2eee4efc-96ce-454a-a92d-d490b9dcbaef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27cc2423d2240ae97a9f890ef55b2b8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:42.684685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1067/2086] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:42 2022] DELETE /identity/v3/users/e27cc2423d2240ae97a9f890ef55b2b8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:43.143623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:43.146390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:43.161235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4be894-a64b-4b2a-b7fa-c1efa652046c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:43.230349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:43.231132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.231419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.231836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] PATH_INFO: `/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.233160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce77caef3b54ea0a551be9f2c89ae3c)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:43.235276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:43.300218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:43.346416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce77caef3b54ea0a551be9f2c89ae3c', 'request_id': 'req-53c6b1b4-46a7-428e-95e1-7449bd867b83'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:43.352525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce77caef3b54ea0a551be9f2c89ae3c', 'request_id': 'req-53c6b1b4-46a7-428e-95e1-7449bd867b83'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:43.456472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce77caef3b54ea0a551be9f2c89ae3c', 'request_id': 'req-53c6b1b4-46a7-428e-95e1-7449bd867b83'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:43.469946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] The token cache is being invalidate because project dce77caef3b54ea0a551be9f2c89ae3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:43.470244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:43.470899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-53c6b1b4-46a7-428e-95e1-7449bd867b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:43.472789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1020/2087] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:43 2022] DELETE /identity/v3/projects/dce77caef3b54ea0a551be9f2c89ae3c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:43.478675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8042-c855-444e-92af-885d85a1524f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.479114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8042-c855-444e-92af-885d85a1524f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.479560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8042-c855-444e-92af-885d85a1524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.503750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-38bb8042-c855-444e-92af-885d85a1524f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:43.505045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-38bb8042-c855-444e-92af-885d85a1524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:43.571116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1068/2088] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:43.578733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.579021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.579295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.602114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:43.603389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:43.611432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1021/2089] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:43.615742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8042-c855-444e-92af-885d85a1524f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:43.626284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8042-c855-444e-92af-885d85a1524f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:43.632340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:43.632612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.632901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.633277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.633941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:43.634247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:43.635854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:43.648342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1069/2090] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:43.652774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:43.662034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153391e0-ca4a-4157-a00e-a8c2e8da5142 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:43.666828 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:43.667428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.667705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.667976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.668645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:43.670544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:43.766441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1022/2091] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:43 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:43.770756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:43.782739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51097ffc-8585-4e43-a86f-155fc0cd4226 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:43.843413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:43.844018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.844320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.844592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.845288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:43.847332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:43.971597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1070/2092] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:43 2022] POST /identity/v3/users => generated 279 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:43.975927 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:43.986313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430cf6a-26de-44e0-9f74-7bf1facac5c8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:43.991418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:43.992136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:43.992477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:43.992779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:43.997760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:43.997760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.005995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1023/2093] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:44.011038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.026449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac9fa-efc7-42ad-b2c3-85673db609b5 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.026814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.027416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.027699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.028012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.028677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.030306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c6520-cd62-4816-904f-937ea3c35c08 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.041397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1071/2094] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:44.045716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.056502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ae5c-d8f7-45c1-876c-90808af1b232 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.061048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.061715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.061998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.062355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] PATH_INFO: `/v3/projects/64322fbc61474725b79d55673472d931/users/d4191d51dd614e198818db74c93364a0/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.073578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:44.078294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64322fbc61474725b79d55673472d931, user_id=d4191d51dd614e198818db74c93364a0, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.081082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.149098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:44.155950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1024/2095] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:44 2022] PUT /identity/v3/projects/64322fbc61474725b79d55673472d931/users/d4191d51dd614e198818db74c93364a0/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:44.167673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.167949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.168257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.197907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] MFA Rules not processed for user `d4191d51dd614e198818db74c93364a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:44.199158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:44.266790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1072/2096] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:44.271453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.338986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d83ed-640f-4cd9-ae76-f2579094d84b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.343992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.344626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.344907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.345195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.345823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.347761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.447134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1025/2097] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:44 2022] POST /identity/v3/projects => generated 357 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:07:44.451342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.462147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026c24-932b-4d2d-b5f9-e51f1922070d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.523170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.523853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.524068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.524412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.525042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.527042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.617279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1073/2098] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:44 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:44.619600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.630861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36c561-049e-48cb-bb10-abd2a9c01897 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.638137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.645580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.645580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.645580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.645580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.645580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.656429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1026/2099] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:44.657621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.668952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c8c46-f486-49a2-9351-08598fa81b21 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.674002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.674887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.675167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.676343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] PATH_INFO: `/v3/projects/45e7efa7a2074b95993fe6dc21a91be8/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.691446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e7efa7a2074b95993fe6dc21a91be8, user_id=ceed5dccf7ca4c389ff6b2f2e6b20c3d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.694858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.800297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1074/2100] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:44 2022] PUT /identity/v3/projects/45e7efa7a2074b95993fe6dc21a91be8/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:44.801900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.813879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6280b4-ae21-4e34-97c2-9e447a99f6b8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.870071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.872048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:44.887227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.887859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.888140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.888437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.889084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.890678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.901292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1027/2101] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:44.904780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:44.914351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf72044-2ece-4654-ab91-0a8470248563 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.918981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.919636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.919924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.920225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] PATH_INFO: `/v3/projects/45e7efa7a2074b95993fe6dc21a91be8/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.922376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e7efa7a2074b95993fe6dc21a91be8, user_id=ceed5dccf7ca4c389ff6b2f2e6b20c3d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:44.925048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74610e-5329-4051-9e47-1dc807a113da admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:44.937375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1028/2102] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:44 2022] PUT /identity/v3/projects/45e7efa7a2074b95993fe6dc21a91be8/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:07:44.945209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a8776-9693-4b74-9c77-8ce7a539c0e1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:44.954752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.955048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.955321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.983411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] MFA Rules not processed for user `ceed5dccf7ca4c389ff6b2f2e6b20c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:44.984650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:44.990414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:44.991745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:44.992044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:44.992349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:44.994100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:45.055955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1029/2103] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:45.063816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:45.065448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:45.067286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:45.069029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:45.075848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1075/2104] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:45.080925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914373e5-d35a-43c3-8951-07b1851eff45 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:45.085353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:45.085959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:45.086399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:45.086675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:45.096227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:45.097928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:45.108109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1030/2105] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:45.365206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:45.385935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:45.394499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dbaab-884d-4a37-a86d-443161a50c42 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:45.398162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:45.398760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:45.399041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:45.399312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:45.465016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:45.466578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5ba77-28dd-4983-b227-04dddc235231 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:45.477260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1076/2106] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:45.948862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:45.955039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:45.966622 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:46.077356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b3e2-b8f0-4e46-a062-e846006c9446 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:46.082084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:46.083229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:46.083510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:46.083783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] PATH_INFO: `/v3/users/4ee8348c14094a7187efd6dd355e3f5f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:46.086961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee8348c14094a7187efd6dd355e3f5f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:46.089112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:46.313564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee8348c14094a7187efd6dd355e3f5f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:46.314159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee8348c14094a7187efd6dd355e3f5f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:46.321540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee8348c14094a7187efd6dd355e3f5f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:46.385564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee8348c14094a7187efd6dd355e3f5f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:46.386203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee8348c14094a7187efd6dd355e3f5f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:46.404390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1031/2107] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:07:45 2022] DELETE /identity/v3/users/4ee8348c14094a7187efd6dd355e3f5f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:46.408909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:46.409193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:46.409468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:46.429379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:46.430605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:46.489270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1077/2108] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:46.499276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:46.500719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:46.557394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e870c4f-764c-4150-91b7-1a731b0efcb5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:46.561160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:46.561762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:46.562042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:46.562316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:46.581722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:46.583191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901433b-fe58-41f1-bcb9-69d2d29ec9c4 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:46.593863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1032/2109] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:46.759759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:47.073416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:47.075313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:47.098237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1b795-080d-46f4-b246-7e7db628cf94 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:47.111743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:47.112437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.112758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.117819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] PATH_INFO: `/v3/projects/f122aa8ceea64d2f8f57230a8a00b0d4` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.117819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f122aa8ceea64d2f8f57230a8a00b0d4)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:47.117819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:47.282735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f122aa8ceea64d2f8f57230a8a00b0d4', 'request_id': 'req-01397e35-0c74-4949-a658-63a23616d4a5'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:47.289195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f122aa8ceea64d2f8f57230a8a00b0d4', 'request_id': 'req-01397e35-0c74-4949-a658-63a23616d4a5'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:47.295986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f122aa8ceea64d2f8f57230a8a00b0d4', 'request_id': 'req-01397e35-0c74-4949-a658-63a23616d4a5'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:47.398999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] The token cache is being invalidate because project f122aa8ceea64d2f8f57230a8a00b0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:07:47.399284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:47.399913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-01397e35-0c74-4949-a658-63a23616d4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:07:47.401463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1078/2110] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:07:47 2022] DELETE /identity/v3/projects/f122aa8ceea64d2f8f57230a8a00b0d4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:47.408511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.409025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.409025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.430417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:47.434760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:47.495593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1033/2111] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:47.503242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.503528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.503872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.523967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:47.525220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:47.533425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1079/2112] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:07:47.541976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:47.552265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0ff20-82bb-471d-8c3f-9f15038768a6 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:47.558027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:47.558685 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.558980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.559274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.559917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:07:47.560254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:47.561894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:47.574792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1034/2113] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:07:47.586792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:47.596962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd5d97-b774-4278-8fcc-3c58fb95886e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:47.602384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:47.602987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.603267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.603537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.604257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:47.606133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:47.777469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1080/2114] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:47 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:47.781440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:47.786105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:47.792374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf32957-435d-41c2-ad73-35961d8240b5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:47.849339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:47.849977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.850350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.851502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.852193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:47.854280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:47.890553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:47.959547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1035/2115] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:47 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:47.964419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:47.975874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12235-ff17-43ec-a311-a02253c88247 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:47.980813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:47.981420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:47.981724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:47.981997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:47.982640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:47.984311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:47.994350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1081/2116] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:47.998769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.009796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588e94-48f4-4fcc-8376-a1b45f8e56ee admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.014667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.015270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.015581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.015858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.016537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.018136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09e9fc-2e35-457a-8c4f-6c90bb7f787f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.031723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1036/2117] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:48.035647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.046294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ec27-15e5-4827-8a7b-1732a455a271 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.051307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.052056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.052380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.052654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] PATH_INFO: `/v3/projects/8a6d211a89bd43959e2d8ab159bb975a/users/b5f6395b6a6b4e449e319f7367aa4d82/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.067514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d211a89bd43959e2d8ab159bb975a, user_id=b5f6395b6a6b4e449e319f7367aa4d82, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.071237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.151929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1082/2118] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:48 2022] PUT /identity/v3/projects/8a6d211a89bd43959e2d8ab159bb975a/users/b5f6395b6a6b4e449e319f7367aa4d82/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:48.162214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.162497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.162809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.189088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] MFA Rules not processed for user `b5f6395b6a6b4e449e319f7367aa4d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:48.190286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:48.237429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:48.251408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1037/2119] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:48.255808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.315650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82565-2a51-4c91-8ee5-e7b75823f625 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.320986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.321593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.321873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.322176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.322801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.327458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.429484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1083/2120] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:48 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:48.431393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.445645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2f01-5f6b-4469-b95d-489a4b533950 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.502635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.510896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.510896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.510896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.510896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.510896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.648129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1038/2121] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:48 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:48.649504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.659579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724893c-1bd7-4e60-baf0-7c4351d38017 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.667467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.668087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.668918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.668918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.669541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.671144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.683180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1084/2122] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:48.687925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.698871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749a09d-54b7-4f3d-ac0b-595253ac9617 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.703911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.704705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.704960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.705250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.706013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.707654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07420d9d-f33f-42fe-9802-f527f3a6b217 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.718240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1039/2123] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:48.722495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:48.733416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c65c3-3889-40f5-bf2e-99c3a9eadce4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:48.737984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:48.738631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.738904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.739236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] PATH_INFO: `/v3/projects/fd0133b316aa454ba3843476f75390da/users/f9b8b8d5665b4def950575a3ed8c9beb/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.757745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0133b316aa454ba3843476f75390da, user_id=f9b8b8d5665b4def950575a3ed8c9beb, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:48.760499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:48.896592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1085/2124] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:48 2022] PUT /identity/v3/projects/fd0133b316aa454ba3843476f75390da/users/f9b8b8d5665b4def950575a3ed8c9beb/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:48.908280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:48.908576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:48.908857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:48.944394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] MFA Rules not processed for user `f9b8b8d5665b4def950575a3ed8c9beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:48.945677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:49.012482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1040/2125] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:49.017065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.086246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497867a-d6da-4ee0-83cd-dd9bc37db365 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.091369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.092474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.092474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.092625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.093225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.095123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.193474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:49.200706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1086/2126] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:07:49 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:49.207953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.218937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680972f-6b6f-42cd-be56-7ff378bc2217 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.274115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.274724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.275004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.275277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.275933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.277895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.393850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1041/2127] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:07:49 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:07:49.398576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.409291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e52595-0ced-4557-9149-9d2115c60cf6 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.414795 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.415497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.415780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.416057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.416744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.418339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.429083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1087/2128] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:49.433270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.443220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc68-d191-4557-bdf0-1800edaf7e47 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.448389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.451898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.451898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.451898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] PATH_INFO: `/v3/projects/aff17ba576a84cdeba272f0cd23d64b1/users/d0d15160f05a408cb2cb61f9c2cd000c/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.467105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff17ba576a84cdeba272f0cd23d64b1, user_id=d0d15160f05a408cb2cb61f9c2cd000c, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.469859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.617550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1042/2129] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:49 2022] PUT /identity/v3/projects/aff17ba576a84cdeba272f0cd23d64b1/users/d0d15160f05a408cb2cb61f9c2cd000c/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:49.625393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.636473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadc654-9c18-49b1-9a12-ad195baf40b6 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.695438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.696206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.696488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.696761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.697403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.698987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4df773-3935-4695-a3a2-edda51a95ac5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.709055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1088/2130] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:07:49.713705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.726676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828742-8cea-4b92-b90a-dc25e3548d51 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.733493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.735336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.735336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.735336 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] PATH_INFO: `/v3/projects/aff17ba576a84cdeba272f0cd23d64b1/users/d0d15160f05a408cb2cb61f9c2cd000c/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.736882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff17ba576a84cdeba272f0cd23d64b1, user_id=d0d15160f05a408cb2cb61f9c2cd000c, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:49.739585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:49.750897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1043/2131] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:07:49 2022] PUT /identity/v3/projects/aff17ba576a84cdeba272f0cd23d64b1/users/d0d15160f05a408cb2cb61f9c2cd000c/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:07:49.761721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.762007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.762375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:49.791979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] MFA Rules not processed for user `d0d15160f05a408cb2cb61f9c2cd000c`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:07:49.793384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:49.864867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1089/2132] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:07:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:07:49.876931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:49.878700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:49.937651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd72a0-d943-41db-a278-b02296116fd6 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:49.941357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:49.941968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:49.942253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:49.942557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:50.003497 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:50.004926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:50.014683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1044/2133] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:07:51.154021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:52.066605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:52.923588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:52.925001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:07:52.937502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff548e-ec4c-489e-a5c8-c1a8031a1b31 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:52.941060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:52.941660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:52.941935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:52.942202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:52.998121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:52.999533 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8475e-8c85-4739-bc9d-b049f8360126 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:53.009257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1090/2134] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:53.254349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:54.467661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:54.819617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:07:54.826325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556c699-a93c-4814-8b7a-463c2a7918f4 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:07:54.830140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:07:54.830749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:07:54.831029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:07:54.831303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:07:54.841508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:07:54.843001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:07:54.852709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1045/2135] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:07:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:07:55.153424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:55.484932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:55.781703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:56.124975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:56.267852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:56.764600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:56.793186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:56.968882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.016181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.136441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.233396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.485087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.656590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.684909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.790558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:57.936688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:58.022261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:58.947804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:59.136571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:07:59.719228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:00.193095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:00.343282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:00.443447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:00.585411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f3d3-f16d-483c-9bb1-93eed0995e40 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:00.585597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f3d3-f16d-483c-9bb1-93eed0995e40 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:00.585854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f3d3-f16d-483c-9bb1-93eed0995e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:00.605944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1a29f3d3-f16d-483c-9bb1-93eed0995e40 None None] MFA Rules not processed for user `1f2a411b051147e58a94dfa03413539b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:00.607123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1a29f3d3-f16d-483c-9bb1-93eed0995e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:00.618045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:00.672213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1091/2136] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:01.118388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:01.443694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:02.621876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:02.702191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.077194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.370377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.437519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.614530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.648590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.911515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:03.968808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.071783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.102298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.438618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.467459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.587460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:04.620000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:07.122843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:07.136489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:07.199696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bcf4c-5bda-40fa-a932-627e6584b7f0 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:07.205459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:07.206125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:07.206401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:07.206675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] PATH_INFO: `/v3/users/b5f6395b6a6b4e449e319f7367aa4d82` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:07.207854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f6395b6a6b4e449e319f7367aa4d82)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:07.210435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:07.490579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f6395b6a6b4e449e319f7367aa4d82'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:07.493620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f6395b6a6b4e449e319f7367aa4d82'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:07.506700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f6395b6a6b4e449e319f7367aa4d82'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:07.576794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f6395b6a6b4e449e319f7367aa4d82'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:07.577875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f6395b6a6b4e449e319f7367aa4d82'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:07.599428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1046/2137] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:07 2022] DELETE /identity/v3/users/b5f6395b6a6b4e449e319f7367aa4d82 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:07.605381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:07.605883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:07.605979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:07.626902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:07.628452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:07.699094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1092/2138] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:07.712177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:07.713643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:07.774697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3182f-a2ee-4d01-8ef2-3ebaa5133b4e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:07.778450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:07.779063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:07.779345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:07.779621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:07.790061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:07.791526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:07.801453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1047/2139] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:08.486437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:08.488025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:08.502677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e8bdb-b77f-41fc-99bd-4497626a1b0a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:08.508590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:08.509237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:08.509516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:08.509789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] PATH_INFO: `/v3/projects/8a6d211a89bd43959e2d8ab159bb975a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:08.511090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6d211a89bd43959e2d8ab159bb975a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:08.513050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:08.620331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:08.623086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6d211a89bd43959e2d8ab159bb975a', 'request_id': 'req-843e7a3b-83e6-4843-81ac-435418b7bc9d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:08.628779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6d211a89bd43959e2d8ab159bb975a', 'request_id': 'req-843e7a3b-83e6-4843-81ac-435418b7bc9d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:08.635798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6d211a89bd43959e2d8ab159bb975a', 'request_id': 'req-843e7a3b-83e6-4843-81ac-435418b7bc9d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:08.745752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] The token cache is being invalidate because project 8a6d211a89bd43959e2d8ab159bb975a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:08.746043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:08.746673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:08.748211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1093/2140] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:08 2022] DELETE /identity/v3/projects/8a6d211a89bd43959e2d8ab159bb975a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:08.752202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:08.762502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3059672-dce6-44a5-9b60-87b765da679f service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:08.794625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:08.818616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:08.819937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:08.820258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:08.826195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] PATH_INFO: `/v3/users/f9b8b8d5665b4def950575a3ed8c9beb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:08.826195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b8b8d5665b4def950575a3ed8c9beb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:08.826195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:09.047576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b8b8d5665b4def950575a3ed8c9beb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.048262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b8b8d5665b4def950575a3ed8c9beb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.055825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b8b8d5665b4def950575a3ed8c9beb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.147230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b8b8d5665b4def950575a3ed8c9beb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.147974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b8b8d5665b4def950575a3ed8c9beb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.167147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1048/2141] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:08 2022] DELETE /identity/v3/users/f9b8b8d5665b4def950575a3ed8c9beb => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:09.703238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:09.705445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:09.719815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e7a3b-83e6-4843-81ac-435418b7bc9d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:09.807541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:09.808211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:09.808497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:09.809082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] PATH_INFO: `/v3/projects/fd0133b316aa454ba3843476f75390da` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:09.810280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0133b316aa454ba3843476f75390da)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:09.812213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:09.954149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0133b316aa454ba3843476f75390da', 'request_id': 'req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.959685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0133b316aa454ba3843476f75390da', 'request_id': 'req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:09.965811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0133b316aa454ba3843476f75390da', 'request_id': 'req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.096405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] The token cache is being invalidate because project fd0133b316aa454ba3843476f75390da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:10.096699 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.097346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.098910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1094/2142] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:09 2022] DELETE /identity/v3/projects/fd0133b316aa454ba3843476f75390da => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:10.103252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:10.104717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:10.117753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e0ca2-e910-4830-82eb-e7bf6ba93363 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:10.173259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:10.173995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:10.174274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:10.174760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] PATH_INFO: `/v3/users/d0d15160f05a408cb2cb61f9c2cd000c` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:10.176013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d15160f05a408cb2cb61f9c2cd000c)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:10.178952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:10.406516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d15160f05a408cb2cb61f9c2cd000c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.407172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d15160f05a408cb2cb61f9c2cd000c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.415103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d15160f05a408cb2cb61f9c2cd000c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.507761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d15160f05a408cb2cb61f9c2cd000c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.508441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d15160f05a408cb2cb61f9c2cd000c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:10.527198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1049/2143] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:10 2022] DELETE /identity/v3/users/d0d15160f05a408cb2cb61f9c2cd000c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:10.535898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:11.092562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:11.098376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:11.109399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb7fb2-bf0c-4dd2-b5d2-bba3003d7cdf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:11.176492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:11.177279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:11.177663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:11.178039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] PATH_INFO: `/v3/projects/aff17ba576a84cdeba272f0cd23d64b1` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:11.180489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff17ba576a84cdeba272f0cd23d64b1)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:11.182511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:11.189171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:11.317828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff17ba576a84cdeba272f0cd23d64b1', 'request_id': 'req-6189bd28-204d-4428-bc9c-5c949b215f24'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:11.323535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff17ba576a84cdeba272f0cd23d64b1', 'request_id': 'req-6189bd28-204d-4428-bc9c-5c949b215f24'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:11.329725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff17ba576a84cdeba272f0cd23d64b1', 'request_id': 'req-6189bd28-204d-4428-bc9c-5c949b215f24'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:11.415389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] The token cache is being invalidate because project aff17ba576a84cdeba272f0cd23d64b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:11.415681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:11.416385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6189bd28-204d-4428-bc9c-5c949b215f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:11.417882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1095/2144] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:11 2022] DELETE /identity/v3/projects/aff17ba576a84cdeba272f0cd23d64b1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:12.417192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:12.418667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:12.444428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c101-a902-49d6-9d43-89c35878765a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:12.509485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:12.510144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:12.510427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:12.510730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] PATH_INFO: `/v3/users/6852322b0f594743b12c9a2ef797abec` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:12.511961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6852322b0f594743b12c9a2ef797abec)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:12.514046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:12.716288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6852322b0f594743b12c9a2ef797abec'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:12.716876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6852322b0f594743b12c9a2ef797abec'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:12.723607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6852322b0f594743b12c9a2ef797abec'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:12.775297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6852322b0f594743b12c9a2ef797abec'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:12.775931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6852322b0f594743b12c9a2ef797abec'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:12.794763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1050/2145] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:12 2022] DELETE /identity/v3/users/6852322b0f594743b12c9a2ef797abec => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:12.799224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:12.799684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:12.799773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:12.819325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:12.820711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:12.881417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1096/2146] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:12.891223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:12.892645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:12.961114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e0fd-f48e-42ce-867c-7c41be1186e9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:12.964991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:12.965637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:12.965898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:12.966169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:12.974079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:12.976907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:12.979317 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5b41-c5bf-4a76-a814-d28f0937f525 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:12.989079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1051/2147] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:13.534837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:13.538583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:13.556209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf597b8-e2b8-4600-8793-051050ac190a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:13.562357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:13.563286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:13.563868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:13.564250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] PATH_INFO: `/v3/projects/2ce095bc88e64797a2fa117e66a88a43` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:13.565508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce095bc88e64797a2fa117e66a88a43)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:13.567711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:13.692279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce095bc88e64797a2fa117e66a88a43', 'request_id': 'req-ed227dff-f03b-41d2-a166-84af56826715'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:13.699773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce095bc88e64797a2fa117e66a88a43', 'request_id': 'req-ed227dff-f03b-41d2-a166-84af56826715'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:13.706611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce095bc88e64797a2fa117e66a88a43', 'request_id': 'req-ed227dff-f03b-41d2-a166-84af56826715'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:13.816679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] The token cache is being invalidate because project 2ce095bc88e64797a2fa117e66a88a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:13.817688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:13.818307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ed227dff-f03b-41d2-a166-84af56826715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:13.821775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1097/2148] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:13 2022] DELETE /identity/v3/projects/2ce095bc88e64797a2fa117e66a88a43 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:13.827005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:13.827527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:13.827778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:13.854666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:13.856471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:13.924663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1052/2149] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:13.932703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:13.933246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:13.933531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:13.953536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:13.954737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:13.963621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1098/2150] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:13.967868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:13.978573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4984c0a-d0cb-4271-9a1f-f872b1ad9c9d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:13.983261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:13.983866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:13.984204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:13.984504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:13.985172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:13.985503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:13.987114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.002371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1053/2151] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:08:14.003556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.013785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c176f5-af65-493b-a873-e8ee3f032421 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.018480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.019109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.019391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.019793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.020602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.023450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.036772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:14.206253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1099/2152] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:14 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:14.211158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.221770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9c3c-2075-4c3e-a7d2-e9753daea337 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.297974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.299094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.299398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.299675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.300347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.302650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.427593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1054/2153] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:14 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:14.430988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.442488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d20b2-6b8a-4898-9391-3b677032f3af admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.447785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.448482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.448883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.449154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.449966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.451723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.462580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1100/2154] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:14.466694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.478146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7d80-6734-48ce-871b-2988b3be0a59 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.482785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.483389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.483669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.483939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.484620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.486231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf9a27-e05d-440e-bc3e-659a956e9f3a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.498096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1055/2155] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:14.502427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.513769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938adb-75ac-4083-aae0-379aa4993b24 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.519609 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.520391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.520704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.521082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] PATH_INFO: `/v3/projects/54df2648916c4681af69062a1d5b7430/users/2927c54860d840538bab1004441c104b/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.540610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54df2648916c4681af69062a1d5b7430, user_id=2927c54860d840538bab1004441c104b, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.543612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.712825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1101/2156] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:14 2022] PUT /identity/v3/projects/54df2648916c4681af69062a1d5b7430/users/2927c54860d840538bab1004441c104b/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:08:14.714585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.715033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.715515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.739173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:14.740543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:14.832289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1056/2157] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:14.844451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:14.853326 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:14.913102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea0a65-4a2c-4e0c-9e26-f35004f5729c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:14.917583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:14.918215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:14.918499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:14.918778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:14.930271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:14.931713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716c22b-fdcc-4915-9c3c-6c247732b935 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:14.942714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1102/2158] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:15.787014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:15.793544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:15.797735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:15.804794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217719fb-18e4-47a2-bda7-ef5fb2c54962 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:15.809697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:15.810456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:15.810807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:15.811091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] PATH_INFO: `/v3/users/d4191d51dd614e198818db74c93364a0` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:15.812452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4191d51dd614e198818db74c93364a0)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:15.814402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:16.035719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4191d51dd614e198818db74c93364a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:16.036941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4191d51dd614e198818db74c93364a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:16.044771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4191d51dd614e198818db74c93364a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:16.153211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4191d51dd614e198818db74c93364a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:16.153856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4191d51dd614e198818db74c93364a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:16.172400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1057/2159] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:15 2022] DELETE /identity/v3/users/d4191d51dd614e198818db74c93364a0 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:16.177034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:16.177516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:16.177638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:16.196880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:16.198078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:16.257236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1103/2160] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:16.267176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:16.268628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:16.325363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22a069-e1bf-4522-a685-5df60613cbc7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:16.329111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:16.329768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:16.330052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:16.330331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:16.340306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:16.341795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:16.352296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1058/2161] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:17.145339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:17.147085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:17.159456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83692e-1ef8-4ed5-9e3a-da24230bc4f3 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:17.165808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:17.165808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:17.165808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:17.165808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] PATH_INFO: `/v3/users/8ebed906710b4faeba23cf7caf973138` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:17.167002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebed906710b4faeba23cf7caf973138)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:17.169035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:17.244967 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:17.251119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:17.268842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b905-7794-45ff-92c1-fb1840be70f9 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:17.274106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:17.274738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:17.281374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:17.281374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] PATH_INFO: `/v3/projects/64322fbc61474725b79d55673472d931` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:17.281374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64322fbc61474725b79d55673472d931)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:17.281374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:17.500888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebed906710b4faeba23cf7caf973138'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.501931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebed906710b4faeba23cf7caf973138'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.504569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64322fbc61474725b79d55673472d931', 'request_id': 'req-6c2185cc-4a71-412a-8270-df05540529a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.507622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:17.512098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64322fbc61474725b79d55673472d931', 'request_id': 'req-6c2185cc-4a71-412a-8270-df05540529a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.512274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebed906710b4faeba23cf7caf973138'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.599429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64322fbc61474725b79d55673472d931', 'request_id': 'req-6c2185cc-4a71-412a-8270-df05540529a0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.610762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:17.615068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] The token cache is being invalidate because project 64322fbc61474725b79d55673472d931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:17.615365 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.616806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.618331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1059/2162] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:17 2022] DELETE /identity/v3/projects/64322fbc61474725b79d55673472d931 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:17.625886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:17.630384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:17.708964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebed906710b4faeba23cf7caf973138'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.712870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b6e98944-e3c8-4454-a220-e5e7d0bf36b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebed906710b4faeba23cf7caf973138'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:17.745943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2185cc-4a71-412a-8270-df05540529a0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:17.755353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:17.756351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:17.757140 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:17.757938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1104/2163] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:17 2022] DELETE /identity/v3/users/8ebed906710b4faeba23cf7caf973138 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:17.759710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] PATH_INFO: `/v3/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:17.761055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceed5dccf7ca4c389ff6b2f2e6b20c3d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:17.762554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:17.763056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:17.763556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:17.763997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:17.787684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:17.789266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:17.860532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1105/2164] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:17.870434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:17.871832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:17.942446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0f48f-10da-4a22-9921-914ec4a467ac None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:17.947309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:17.947946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:17.948264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:17.948537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:17.957720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:17.959134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:17.969099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1106/2165] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:08:18.041331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceed5dccf7ca4c389ff6b2f2e6b20c3d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:18.043740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceed5dccf7ca4c389ff6b2f2e6b20c3d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:18.068027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceed5dccf7ca4c389ff6b2f2e6b20c3d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:18.218224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceed5dccf7ca4c389ff6b2f2e6b20c3d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:18.219635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceed5dccf7ca4c389ff6b2f2e6b20c3d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:18.254290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1060/2166] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:17 2022] DELETE /identity/v3/users/ceed5dccf7ca4c389ff6b2f2e6b20c3d => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:18.842937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:18.846721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:18.888575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99bb87-7fc1-4415-b6fb-d8285db2a264 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:18.991542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:18.991542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:18.991542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:18.991542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] PATH_INFO: `/v3/projects/6e5dcb22055547d29deaf8c50f377778` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:18.992167 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5dcb22055547d29deaf8c50f377778)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:18.995319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:19.089504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:19.089610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:19.107554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647223-c295-4f06-8db8-75a10e619a3b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:19.112538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:19.113199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.113491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.113760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] PATH_INFO: `/v3/projects/45e7efa7a2074b95993fe6dc21a91be8` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.114918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e7efa7a2074b95993fe6dc21a91be8)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:19.117426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:19.251827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5dcb22055547d29deaf8c50f377778', 'request_id': 'req-a68f0819-26ca-4720-8cb3-0e254e28c1cf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.258690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5dcb22055547d29deaf8c50f377778', 'request_id': 'req-a68f0819-26ca-4720-8cb3-0e254e28c1cf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.266447 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5dcb22055547d29deaf8c50f377778', 'request_id': 'req-a68f0819-26ca-4720-8cb3-0e254e28c1cf'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.317161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e7efa7a2074b95993fe6dc21a91be8', 'request_id': 'req-93f06592-af2e-4834-b15a-d435801ee572'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.323701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e7efa7a2074b95993fe6dc21a91be8', 'request_id': 'req-93f06592-af2e-4834-b15a-d435801ee572'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.370617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] The token cache is being invalidate because project 6e5dcb22055547d29deaf8c50f377778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:19.371083 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.371978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.386315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1107/2167] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:18 2022] DELETE /identity/v3/projects/6e5dcb22055547d29deaf8c50f377778 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:19.393987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:19.393987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:19.495369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e7efa7a2074b95993fe6dc21a91be8', 'request_id': 'req-93f06592-af2e-4834-b15a-d435801ee572'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.513653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] The token cache is being invalidate because project 45e7efa7a2074b95993fe6dc21a91be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:19.514337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.515224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-93f06592-af2e-4834-b15a-d435801ee572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.520571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1061/2168] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:19 2022] DELETE /identity/v3/projects/45e7efa7a2074b95993fe6dc21a91be8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:19.523671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cfc84-3834-48ae-973b-24a160fcb238 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.523889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0819-26ca-4720-8cb3-0e254e28c1cf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:19.525019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cfc84-3834-48ae-973b-24a160fcb238 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.525851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cfc84-3834-48ae-973b-24a160fcb238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.553981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-636cfc84-3834-48ae-973b-24a160fcb238 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:19.555576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-636cfc84-3834-48ae-973b-24a160fcb238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:19.604356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:19.605106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.605393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.605668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] PATH_INFO: `/v3/users/b944a42b78a24159a05cef0589527ed8` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.606842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b944a42b78a24159a05cef0589527ed8)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:19.608893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:19.668232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1062/2169] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:19.677532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.677819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.678198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.704516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:19.704516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:19.713247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1063/2170] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:19.714318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:19.723929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a370-cadb-4a6d-98ec-5fc63c81889e None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:19.730924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:19.732713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.733003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.733286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.737543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:19.738274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:19.741695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:19.756741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1064/2171] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:08:19.760708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:19.787003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533068f9-8db7-4f3e-b3ef-bcb68edfc25d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:19.788054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:19.789449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:19.789808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:19.790402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:19.803386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:19.803386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:19.872449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b944a42b78a24159a05cef0589527ed8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.873867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b944a42b78a24159a05cef0589527ed8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.907913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b944a42b78a24159a05cef0589527ed8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:19.952247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1065/2172] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:19 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:19.955586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.033655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b944a42b78a24159a05cef0589527ed8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:20.034573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b944a42b78a24159a05cef0589527ed8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:20.068779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afd05-2653-435f-946d-8f03a97003f5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:20.073261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1108/2173] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:19 2022] DELETE /identity/v3/users/b944a42b78a24159a05cef0589527ed8 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:20.079586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:20.080487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.080950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.081819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.083163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:20.086411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:20.243472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1066/2174] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:19 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:20.247979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.250943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:20.360236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd15528-5f34-4af4-befe-dcce9b3f4bb0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:20.367820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:20.367820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.367820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.367820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.368259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:20.370258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:20.383532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1109/2175] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:20.387309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.398380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3bcc1-d1f8-4251-b711-f244a5d1c1d5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:20.403712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:20.404396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.404659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.404934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.405744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:20.407395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44947a01-2f06-4365-a08e-b5146fca9b8c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:20.444546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1067/2176] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:20.447217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.460250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00a78-23a9-4d6c-a985-865811afc66a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:20.486451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:20.487145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.487429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.487775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] PATH_INFO: `/v3/projects/9015a9020cdb44e2be8fd3b0013f449d/users/811ae360e43a4d8db97150447e4114eb/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.508444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015a9020cdb44e2be8fd3b0013f449d, user_id=811ae360e43a4d8db97150447e4114eb, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:20.514149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:20.628793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1110/2177] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:20 2022] PUT /identity/v3/projects/9015a9020cdb44e2be8fd3b0013f449d/users/811ae360e43a4d8db97150447e4114eb/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:20.639763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.640076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.640438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.676377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] MFA Rules not processed for user `811ae360e43a4d8db97150447e4114eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:20.677817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:20.760381 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1068/2178] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:20.765108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.906886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:20.911932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd1c20-529c-4943-9a66-7c18c69ef1be admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:20.918389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:20.919572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:20.920064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:20.920580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:20.921433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:20.924047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:20.966818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:20.969049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:20.999364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f64f-d21b-4db9-aa67-c5d6f3e3caaf None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:21.020838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:21.025194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.025693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.026121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] PATH_INFO: `/v3/projects/aaa6621ccd034c48983cc72635c4c387` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.029493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa6621ccd034c48983cc72635c4c387)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:21.031626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:21.080332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1111/2179] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:20 2022] POST /identity/v3/projects => generated 387 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:21.081597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:21.091171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:21.225658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddad77-dfa2-4306-a394-d813cee9f43a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:21.232473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:21.233225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.233603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.233966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.235046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:21.241997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:21.689925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa6621ccd034c48983cc72635c4c387', 'request_id': 'req-58caec51-ffa1-47b2-8b74-431cfa1cbc51'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:21.696277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa6621ccd034c48983cc72635c4c387', 'request_id': 'req-58caec51-ffa1-47b2-8b74-431cfa1cbc51'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:21.746849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1112/2180] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:21 2022] POST /identity/v3/users => generated 300 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:21.750719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:21.764353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f629ed-2d28-46bb-a986-4f3f1d7bb718 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:21.769437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa6621ccd034c48983cc72635c4c387', 'request_id': 'req-58caec51-ffa1-47b2-8b74-431cfa1cbc51'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:21.784384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] The token cache is being invalidate because project aaa6621ccd034c48983cc72635c4c387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:21.784947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:21.787840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-58caec51-ffa1-47b2-8b74-431cfa1cbc51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:21.793101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1069/2181] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:20 2022] DELETE /identity/v3/projects/aaa6621ccd034c48983cc72635c4c387 => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:08:21.794699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2bee-201b-486a-a306-4c4b5d008e22 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.795164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2bee-201b-486a-a306-4c4b5d008e22 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.796002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2bee-201b-486a-a306-4c4b5d008e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.822518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e17c2bee-201b-486a-a306-4c4b5d008e22 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:21.824354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e17c2bee-201b-486a-a306-4c4b5d008e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:21.852930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:21.853601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.854407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.854697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.855370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:21.857148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:21.880002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1113/2182] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:21.885078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:21.888382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:21.903241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9471fc1-5d5b-4f87-91c1-6f5d55b5020d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:21.908786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:21.910790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.911248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.912874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] PATH_INFO: `/v3/projects/806572149ea743478a0418cfa131efed/users/b796726f8cdd4646a0676b0e5dba2301/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.919201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1070/2183] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:21.929592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.931498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.931498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.935183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=806572149ea743478a0418cfa131efed, user_id=b796726f8cdd4646a0676b0e5dba2301, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:21.938340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:21.960128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:21.961765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:21.970476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1071/2184] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:21.974719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:21.984846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107533a-0128-4d87-a7ea-2fa28f314c6a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:21.989470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:21.990065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:21.990341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:21.990634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:21.991299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:21.991657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:21.993274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.011405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1072/2185] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:08:22.015783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.030524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9296-abe1-415c-9056-c74a61c57345 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.034833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1114/2186] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:21 2022] PUT /identity/v3/projects/806572149ea743478a0418cfa131efed/users/b796726f8cdd4646a0676b0e5dba2301/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:08:22.038600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.171804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.171804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.171804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.171804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.172420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.173529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc146a9-5290-4c17-a39c-62639dc9b5ce admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.180871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.191394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.192041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.200471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.200810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.201537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.208434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.238195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1115/2187] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:22.242880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.256888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5289-47a7-47e2-9a48-c020c48e437b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.265021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.265021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.265021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.265021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] PATH_INFO: `/v3/projects/806572149ea743478a0418cfa131efed/users/b796726f8cdd4646a0676b0e5dba2301/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.265687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806572149ea743478a0418cfa131efed, user_id=b796726f8cdd4646a0676b0e5dba2301, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.268229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42798d-9c99-4293-a384-261750752487 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.298236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1073/2188] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:22 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:22.308229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1116/2189] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:22 2022] PUT /identity/v3/projects/806572149ea743478a0418cfa131efed/users/b796726f8cdd4646a0676b0e5dba2301/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:22.309236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.309478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.309703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.310209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.381811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] MFA Rules not processed for user `b796726f8cdd4646a0676b0e5dba2301`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:22.384175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:22.428753 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613b5-5808-4e88-ad9d-f167527fe87c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.435218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.437149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.437441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.437731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.438377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.441296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.499290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1117/2190] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:22.509272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.510431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:22.542549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1074/2191] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:22 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:22.547933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.594803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcfcd-075e-4792-9166-ba991b63c916 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.613271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.614257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.614698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.615121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.616567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.618796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.643504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1075/2192] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:22.651496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.660618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983163ea-9cbb-44fe-a9f9-72f50e7bd83a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.665737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.666490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.666854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.667199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.672939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1ba97-1f62-48e8-9cc4-c5c9a79bc4c8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.682109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.682739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.683026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.683358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.684105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.685794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.686175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.687676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.707816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1118/2193] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:08:22.711808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1076/2194] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:22.715645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.731262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bfcda-d73f-45b9-882f-87c6f0cc17a1 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.740899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:22.742415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.753004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.753476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] PATH_INFO: `/v3/projects/da965a7c574642ddad7ccd5fbae18e88/users/3490777eb12e4c84a097197d86bf08df/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.784485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da965a7c574642ddad7ccd5fbae18e88, user_id=3490777eb12e4c84a097197d86bf08df, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:22.795623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268f32d-73e5-41b4-b614-95e65e91b9c6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:22.880639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1119/2195] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:22 2022] PUT /identity/v3/projects/da965a7c574642ddad7ccd5fbae18e88/users/3490777eb12e4c84a097197d86bf08df/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:22.889111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:22.907285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:22.910197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:22.912927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:22.920980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b02542-408d-4270-9f14-8adce3469dea admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:22.984655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] MFA Rules not processed for user `3490777eb12e4c84a097197d86bf08df`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:22.986242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.052118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.052824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.053532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.055519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] PATH_INFO: `/v3/users/811ae360e43a4d8db97150447e4114eb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.056946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] RBAC: Authorizing `identity:delete_user(user_id=811ae360e43a4d8db97150447e4114eb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.059107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.089586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1120/2196] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:23.091140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.106984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a919a17-fa43-449c-a3d1-95ab72d85f46 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.114724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.116450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.116905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.117360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.118368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.120682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.271237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1121/2197] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:23 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:23.280890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.391099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6071b-ac09-4747-81a3-e6d1c021397e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.399934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811ae360e43a4d8db97150447e4114eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:23.401224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811ae360e43a4d8db97150447e4114eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:23.406656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.407563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.409951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.410476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.411488 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.414431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.418024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811ae360e43a4d8db97150447e4114eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:23.638523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811ae360e43a4d8db97150447e4114eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:23.646342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-986623f7-d4fd-4532-b45d-82ff40f8225b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811ae360e43a4d8db97150447e4114eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:23.659871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1077/2198] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:22 2022] DELETE /identity/v3/users/811ae360e43a4d8db97150447e4114eb => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:23.664873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da717df-5c2f-497e-befb-d73e910d3c43 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.665169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da717df-5c2f-497e-befb-d73e910d3c43 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.665449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da717df-5c2f-497e-befb-d73e910d3c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.692220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4da717df-5c2f-497e-befb-d73e910d3c43 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:23.692220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4da717df-5c2f-497e-befb-d73e910d3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.757254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1078/2199] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:23.759413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.759916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.760344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.792532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] MFA Rules not processed for user `2927c54860d840538bab1004441c104b`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:23.801661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.809075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1122/2200] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:23 2022] POST /identity/v3/users => generated 299 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:23.810073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.811811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.866532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1079/2201] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:23.867470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.868779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.875002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4e9c-919b-4483-ab12-de0f96da6722 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.883207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a103f-bbed-41e4-ae46-af6216e73dca None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.887755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.888383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.888688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.888963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.889607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.891257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.901934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1080/2202] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:23.902818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.907869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:23.921915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6af6-9094-4ad6-83a1-f1e9dc32f917 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.926781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.927408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.927683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.928802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.929212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.931108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.940906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.941603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.941897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.942175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:23.953244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:23.954462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:23.964714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1123/2203] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:23.965514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:23.975505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097f515-5d1c-4da7-ba5c-b4cc2d7dfc61 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:23.980471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:23.981139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:23.981475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:23.981742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] PATH_INFO: `/v3/projects/6c034e3d2b7345268b35c2ddd7065015/users/3be87758ddbb47978bdfdc7d285d736b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.000105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c034e3d2b7345268b35c2ddd7065015, user_id=3be87758ddbb47978bdfdc7d285d736b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.002947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.038273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:24.170027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1081/2204] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:23 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:24.171177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.232273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e0363-4d40-455b-9f2c-78efdca71532 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.237214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.237894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.238178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.238459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.239107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.241972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.290540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1124/2205] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:23 2022] PUT /identity/v3/projects/6c034e3d2b7345268b35c2ddd7065015/users/3be87758ddbb47978bdfdc7d285d736b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:24.294483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.364938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13a4e-89b0-4ed8-a8ea-a62adbce1c73 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.372175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.372175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.372175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.372175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.372917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.375275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.388186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1125/2206] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:24.391749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.403188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c656c1e-62e1-47b7-8342-04406f95fa22 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.408089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.408778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.409055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.409355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] PATH_INFO: `/v3/projects/6c034e3d2b7345268b35c2ddd7065015/users/3be87758ddbb47978bdfdc7d285d736b/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.411690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c034e3d2b7345268b35c2ddd7065015, user_id=3be87758ddbb47978bdfdc7d285d736b, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.415076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec0c21-6e6c-46e8-89f1-a5cc4de28d28 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.427274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1126/2207] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:24 2022] PUT /identity/v3/projects/6c034e3d2b7345268b35c2ddd7065015/users/3be87758ddbb47978bdfdc7d285d736b/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:08:24.439547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.439834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.440111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.473885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] MFA Rules not processed for user `3be87758ddbb47978bdfdc7d285d736b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:24.484191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1082/2208] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:24 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:24.484354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:24.487728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.498340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358279e-08a0-4073-9e87-9b2138814775 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.572355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.573564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.573681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.574142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.574593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.577384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.595849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1083/2209] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:24.602689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.619185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50c75-41f2-416d-8f01-6aaf05e1a218 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.621779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1127/2210] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:24.637127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.637335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.637915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.638371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.638659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] PATH_INFO: `/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573/users/146e0804900a41598c1fd5b4508d7b88/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.638941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:24.662321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfe6b887aa041b7a0de4ce722cb1573, user_id=146e0804900a41598c1fd5b4508d7b88, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.665867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.713315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eef91-5c31-4e9d-b815-3f42b2620f53 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.717932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.718643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.718984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.719406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.721290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:24.768978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1084/2211] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:24 2022] PUT /identity/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573/users/146e0804900a41598c1fd5b4508d7b88/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:24.768978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.773941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:24.846411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:24.874029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e82a-7b39-4417-b010-094796108d94 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.874575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.876262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.881725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.882624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.883105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.883563 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] PATH_INFO: `/v3/projects/9015a9020cdb44e2be8fd3b0013f449d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.885153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9015a9020cdb44e2be8fd3b0013f449d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.887285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.888921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1128/2212] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:24.889991 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.919437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff705091-bf1c-4eeb-916a-9c22f3dfe032 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.920105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.920921 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.921475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.921756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.922584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.924313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:24.937593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1129/2213] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:24.942701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:24.967759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4cd82-dad4-4f5f-8000-3704b941c87e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:24.981897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:24.984059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:24.984059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:24.984059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] PATH_INFO: `/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573/users/146e0804900a41598c1fd5b4508d7b88/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:24.993347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfe6b887aa041b7a0de4ce722cb1573, user_id=146e0804900a41598c1fd5b4508d7b88, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:24.999358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:25.026988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1130/2214] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:24 2022] PUT /identity/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573/users/146e0804900a41598c1fd5b4508d7b88/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:25.080799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9015a9020cdb44e2be8fd3b0013f449d', 'request_id': 'req-361d76d7-37b2-434d-831d-cfdd5959ce69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.095260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9015a9020cdb44e2be8fd3b0013f449d', 'request_id': 'req-361d76d7-37b2-434d-831d-cfdd5959ce69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.307214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9015a9020cdb44e2be8fd3b0013f449d', 'request_id': 'req-361d76d7-37b2-434d-831d-cfdd5959ce69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.320036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] The token cache is being invalidate because project 9015a9020cdb44e2be8fd3b0013f449d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:25.320582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.321589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.323569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1085/2215] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:24 2022] DELETE /identity/v3/projects/9015a9020cdb44e2be8fd3b0013f449d => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:25.327427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:25.329593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:25.419977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb4d8c-c67a-482f-8bcf-d4d622ee293c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:25.512865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:25.513739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:25.514095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:25.514481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] PATH_INFO: `/v3/users/b796726f8cdd4646a0676b0e5dba2301` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:25.515888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b796726f8cdd4646a0676b0e5dba2301)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:25.524023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:25.781487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b796726f8cdd4646a0676b0e5dba2301'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.782547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b796726f8cdd4646a0676b0e5dba2301'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.791727 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b796726f8cdd4646a0676b0e5dba2301'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.975546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b796726f8cdd4646a0676b0e5dba2301'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:25.976265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ad041977-d13f-4dbd-b083-96447703858a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b796726f8cdd4646a0676b0e5dba2301'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:26.026287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1131/2216] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:25 2022] DELETE /identity/v3/users/b796726f8cdd4646a0676b0e5dba2301 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:08:26.967518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:26.999589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d76d7-37b2-434d-831d-cfdd5959ce69 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:27.074687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:27.076256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:27.076740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:27.077183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] PATH_INFO: `/v3/projects/806572149ea743478a0418cfa131efed` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:27.078767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=806572149ea743478a0418cfa131efed)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:27.082724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:27.368961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806572149ea743478a0418cfa131efed', 'request_id': 'req-006a955e-e93c-427e-8a09-79376e0c08c0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:27.376139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806572149ea743478a0418cfa131efed', 'request_id': 'req-006a955e-e93c-427e-8a09-79376e0c08c0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:27.586996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806572149ea743478a0418cfa131efed', 'request_id': 'req-006a955e-e93c-427e-8a09-79376e0c08c0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:27.590803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:27.634144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] The token cache is being invalidate because project 806572149ea743478a0418cfa131efed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:27.634144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:27.634144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-006a955e-e93c-427e-8a09-79376e0c08c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:27.634144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1086/2217] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:26 2022] DELETE /identity/v3/projects/806572149ea743478a0418cfa131efed => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:27.634756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:27.634756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:27.634756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:27.656220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:27.657498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:27.752052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1132/2218] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:27.760689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:27.760980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:27.761261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:27.783195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:27.792077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:27.801699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1087/2219] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:27.803493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:27.815553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303288d-c36c-44bd-9525-51a724f2a6fb None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:27.831657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:27.831657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:27.831657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:27.831657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:27.833005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:27.833327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:27.835524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:27.852066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1133/2220] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:08:27.853353 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:27.863903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa40a0-e6b5-4e9f-a7e9-b601f28db16f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:27.871103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:27.871902 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:27.872026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:27.872301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:27.873198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:27.876671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:28.044419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:28.063080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1088/2221] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:27 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:28.066774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:28.078291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5611d3-2299-4475-984d-9d5cab994b2f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:28.213394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:28.214204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.214592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.217518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.218239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:28.220330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:28.347208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1134/2222] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:28 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:28.351633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:28.364835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b50098-67bd-4328-b074-7f6856db41af admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:28.373739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:28.374653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.375137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.375593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.376460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:28.378544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:28.404772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1089/2223] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:28.406636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:28.420204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fcb8f-cde2-471b-bf33-a4ed5d2e9a80 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:28.426077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:28.426818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.427159 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.427501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.428349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:28.430159 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ce654-a6ef-408c-9589-16df928bd99f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:28.447331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:28.448242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1135/2224] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:28.460861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0cb3f-45d5-4671-a52d-275fc6547da3 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:28.466129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:28.466798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.467078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.467486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] PATH_INFO: `/v3/projects/7eadacd4eec048d69c41c8423e915e17/users/6266933242a049fb8e0ef6321355f436/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.489235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eadacd4eec048d69c41c8423e915e17, user_id=6266933242a049fb8e0ef6321355f436, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:28.492709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:28.629475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1090/2225] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:28 2022] PUT /identity/v3/projects/7eadacd4eec048d69c41c8423e915e17/users/6266933242a049fb8e0ef6321355f436/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:28.631537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.631843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.632196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.721172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] MFA Rules not processed for user `6266933242a049fb8e0ef6321355f436`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:28.723044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:28.825029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1136/2226] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:28.837040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:28.839076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:28.957525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d9328-be4a-45b1-9b40-3877fe7faa10 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:28.963876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:28.963876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:28.963876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:28.963876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:28.983230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:28.985765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbe9ce-4168-40f8-9ead-c03aaf8fe8f5 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:29.002276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1091/2227] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:08:29.030719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:29.121720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaec6fb-f77c-427b-91d2-89b852524526 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:29.127918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:29.129717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:29.130627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:29.131774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] PATH_INFO: `/v3/users/6266933242a049fb8e0ef6321355f436` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:29.138335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6266933242a049fb8e0ef6321355f436)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:29.138335 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:29.414402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6266933242a049fb8e0ef6321355f436'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:29.415599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6266933242a049fb8e0ef6321355f436'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:29.429832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6266933242a049fb8e0ef6321355f436'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:29.580751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6266933242a049fb8e0ef6321355f436'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:29.581415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6266933242a049fb8e0ef6321355f436'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:29.608203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1137/2228] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:29 2022] DELETE /identity/v3/users/6266933242a049fb8e0ef6321355f436 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:29.610059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:29.617978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:29.617978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:29.633223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:29.634450 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:29.734162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1092/2229] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:29.764682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:29.766352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:29.841767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45b57-3c9b-4d75-91ca-c9a23e0d5ab5 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:29.848975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:29.848975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:29.848975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:29.848975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:29.866014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:29.867971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcd937-413f-4326-85c1-8a52c5ecc3c6 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:29.885337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1138/2230] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:30.640810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:30.643187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:30.658900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c542f4-b6ff-4a73-9115-59f3d334e4e0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:30.664280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:30.665429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:30.665865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:30.666289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] PATH_INFO: `/v3/projects/7eadacd4eec048d69c41c8423e915e17` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:30.667740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eadacd4eec048d69c41c8423e915e17)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:30.674965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:30.808629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eadacd4eec048d69c41c8423e915e17', 'request_id': 'req-1af60c51-200d-48af-9c7d-24708ced3cc6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:30.814404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eadacd4eec048d69c41c8423e915e17', 'request_id': 'req-1af60c51-200d-48af-9c7d-24708ced3cc6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:30.914635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eadacd4eec048d69c41c8423e915e17', 'request_id': 'req-1af60c51-200d-48af-9c7d-24708ced3cc6'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:30.940919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] The token cache is being invalidate because project 7eadacd4eec048d69c41c8423e915e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:30.941222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:30.952175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1af60c51-200d-48af-9c7d-24708ced3cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:30.953835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1093/2231] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:30 2022] DELETE /identity/v3/projects/7eadacd4eec048d69c41c8423e915e17 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:30.958584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:30.959127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:30.960067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:30.985027 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:30.986258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:31.058038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1139/2232] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:31.067013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.067530 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.067804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.091332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:31.092850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:31.105566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1094/2233] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:31.107170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.123355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700861d-d3b4-43b7-83c6-5cdfb5430555 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.129846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.131786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.132337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.132821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.134053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:31.134682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.137637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.159193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.160366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1140/2234] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:08:31.169630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bb114-0d78-4b34-98c9-b493e83fb81f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.174759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.175558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.177139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.177139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.177139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.180370 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.337827 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1095/2235] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:31 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:08:31.344405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.353678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c8a48-a259-4a17-a9d4-6f1d25f1b78e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.421014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.422053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.422407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.422744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.424190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.426285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.559397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1141/2236] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:31 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:31.567950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.583857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24238-05a9-49e6-8bcf-f3499930ea17 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.593573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.595713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.595713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.595713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.601229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.601229 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.612182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1096/2237] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:31.616295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.626445 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71c5ee-ecbc-4ffb-ab80-f54299bb621a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.632463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.633078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.633805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.634081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.634733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.636870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755935c5-9c3d-4bda-b5a0-7918b6140871 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.651310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1142/2238] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:31.652659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.664045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96af3f-8bda-4959-aa71-4e1d5736b6a1 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:31.669670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:31.670684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.671123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.671611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] PATH_INFO: `/v3/projects/98a95555525549a381076118b5431f19/users/9a384e48c8674dd1b926bd1072d65764/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.692101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a95555525549a381076118b5431f19, user_id=9a384e48c8674dd1b926bd1072d65764, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:31.694909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:31.792545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1097/2239] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:31 2022] PUT /identity/v3/projects/98a95555525549a381076118b5431f19/users/9a384e48c8674dd1b926bd1072d65764/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:31.803680 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee00e-d63c-4a20-9350-1d8d62775776 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:31.804002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee00e-d63c-4a20-9350-1d8d62775776 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:31.804286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee00e-d63c-4a20-9350-1d8d62775776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:31.839407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-485ee00e-d63c-4a20-9350-1d8d62775776 None None] MFA Rules not processed for user `9a384e48c8674dd1b926bd1072d65764`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:31.840715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-485ee00e-d63c-4a20-9350-1d8d62775776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:31.936087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1143/2240] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:31.961448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:31.961448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:32.044606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105269a-0a3f-4e12-9df5-f1c80fa7c9ff admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:32.050403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:32.051313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:32.052973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:32.054060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:32.070443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:32.072257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ad744-9e5f-4b76-8947-603aa5073821 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:32.098969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1098/2241] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:32.329558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:32.524394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:32.837864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:32.838304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:32.838755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:32.878547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] MFA Rules not processed for user `7daac509316048ec903a5d78d3586aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:32.879957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:32.981738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1144/2242] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:33.698202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072515b-7748-403d-94ed-245972d0d83c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:33.698741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072515b-7748-403d-94ed-245972d0d83c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:33.698741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072515b-7748-403d-94ed-245972d0d83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:33.734374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f072515b-7748-403d-94ed-245972d0d83c None None] MFA Rules not processed for user `146e0804900a41598c1fd5b4508d7b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:33.738458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f072515b-7748-403d-94ed-245972d0d83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:33.807223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1099/2243] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:33.821197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:33.822672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:33.896255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9cc7-29be-4adb-a70e-6de902d92013 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:33.903697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:33.904420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:33.904707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:33.907846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:33.907846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:34.012111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:34.013948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b780bec-776b-48a0-9c64-13e344f8f48b service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:34.024836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1145/2244] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:34.593155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072515b-7748-403d-94ed-245972d0d83c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:34.605298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072515b-7748-403d-94ed-245972d0d83c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:34.611034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:34.611662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:34.611973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:34.613703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:34.628582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:34.644055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:34.684612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509109cc-7f14-4182-83db-fe6200ad221f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:34.685242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509109cc-7f14-4182-83db-fe6200ad221f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:34.685780 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509109cc-7f14-4182-83db-fe6200ad221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:34.722370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-509109cc-7f14-4182-83db-fe6200ad221f None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:34.725498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-509109cc-7f14-4182-83db-fe6200ad221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:34.771325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:34.778554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:34.805123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1100/2245] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:34.820947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1146/2246] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:34.898369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:34.900000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:35.007884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce97219-6a88-4510-9ce6-feb75740dd17 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:35.014152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:35.014775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:35.015056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:35.015328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:35.025004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:35.027428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:35.054687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1101/2247] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:08:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:08:35.907730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:38.096451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:38.186868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff209a0f-0bb0-4e0a-8994-9c844bfd6d20 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:38.187185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff209a0f-0bb0-4e0a-8994-9c844bfd6d20 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:38.187465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff209a0f-0bb0-4e0a-8994-9c844bfd6d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:38.206692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ff209a0f-0bb0-4e0a-8994-9c844bfd6d20 None None] MFA Rules not processed for user `2927c54860d840538bab1004441c104b`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:38.207943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ff209a0f-0bb0-4e0a-8994-9c844bfd6d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:38.215458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1147/2248] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:39.546259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:39.672283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:40.209449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:40.306312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:42.906044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:43.071939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:45.259761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:45.460871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:48.637717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:49.535856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:50.013771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:50.015416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:50.112110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2028c-0ba4-417e-b9ac-9f852dd6dcc6 service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:50.121026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:50.121985 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:50.122274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:50.122550 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] PATH_INFO: `/v3/users/3490777eb12e4c84a097197d86bf08df` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:50.124086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3490777eb12e4c84a097197d86bf08df)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:50.126524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:50.357641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:50.368616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3490777eb12e4c84a097197d86bf08df'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:50.369300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3490777eb12e4c84a097197d86bf08df'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:50.376599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3490777eb12e4c84a097197d86bf08df'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:50.512323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3490777eb12e4c84a097197d86bf08df'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:50.512990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3490777eb12e4c84a097197d86bf08df'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:50.563514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1102/2249] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:50 2022] DELETE /identity/v3/users/3490777eb12e4c84a097197d86bf08df => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:50.569043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:50.569686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:50.569686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:50.596203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:50.597762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:50.682483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1148/2250] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:50.694436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:50.696241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:50.795036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ffa46-4fea-4ba9-8f21-6f75717971f1 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:50.800854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:50.801314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:50.801645 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:50.801986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:50.817500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:50.819885 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:50.840713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1103/2251] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:51.490371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:51.490371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:51.502865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64043b8e-4b41-4f85-8c6c-33a78ffdf995 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:51.514474 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:51.515182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:51.515457 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:51.515776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] PATH_INFO: `/v3/projects/da965a7c574642ddad7ccd5fbae18e88` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:51.517169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da965a7c574642ddad7ccd5fbae18e88)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:51.519336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:51.704761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da965a7c574642ddad7ccd5fbae18e88', 'request_id': 'req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:51.714339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da965a7c574642ddad7ccd5fbae18e88', 'request_id': 'req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:51.722575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da965a7c574642ddad7ccd5fbae18e88', 'request_id': 'req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:51.826254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] The token cache is being invalidate because project da965a7c574642ddad7ccd5fbae18e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:51.826684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:51.827505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:51.832947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1149/2252] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:51 2022] DELETE /identity/v3/projects/da965a7c574642ddad7ccd5fbae18e88 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:51.834686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:51.849038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afd3f0-ae70-4e88-b2c2-6b47e09b04ba service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:51.936076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:51.937105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:51.937394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:51.937690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] PATH_INFO: `/v3/users/3be87758ddbb47978bdfdc7d285d736b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:51.943293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be87758ddbb47978bdfdc7d285d736b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:51.945412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:52.190253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be87758ddbb47978bdfdc7d285d736b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:52.194612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be87758ddbb47978bdfdc7d285d736b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:52.206701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be87758ddbb47978bdfdc7d285d736b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:52.210493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:52.328964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be87758ddbb47978bdfdc7d285d736b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:52.329909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-665b8fb9-3b32-4aba-bdc1-353ad121a085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be87758ddbb47978bdfdc7d285d736b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:52.363723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1104/2253] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:51 2022] DELETE /identity/v3/users/3be87758ddbb47978bdfdc7d285d736b => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:52.941959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:52.945276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:52.961941 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85eaaf-d90a-48da-8b1e-cdbbd8fd273f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:53.028846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:53.053263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:53.053263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.053263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.053263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] PATH_INFO: `/v3/projects/6c034e3d2b7345268b35c2ddd7065015` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.054469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c034e3d2b7345268b35c2ddd7065015)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:53.054469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:53.238231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c034e3d2b7345268b35c2ddd7065015', 'request_id': 'req-f04b72dd-8916-4f17-9fa8-47c529e1b55a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:53.245520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c034e3d2b7345268b35c2ddd7065015', 'request_id': 'req-f04b72dd-8916-4f17-9fa8-47c529e1b55a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:53.254091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c034e3d2b7345268b35c2ddd7065015', 'request_id': 'req-f04b72dd-8916-4f17-9fa8-47c529e1b55a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:53.319589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] The token cache is being invalidate because project 6c034e3d2b7345268b35c2ddd7065015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:08:53.320235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:53.323275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f04b72dd-8916-4f17-9fa8-47c529e1b55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:53.323644 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1150/2254] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:08:52 2022] DELETE /identity/v3/projects/6c034e3d2b7345268b35c2ddd7065015 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:53.329206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.329814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.330045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.354573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:53.356021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:53.475457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1105/2255] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:53.477657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.478005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.478501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.532659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:53.534522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:53.562890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1151/2256] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:53.568617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:53.598606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17d03-612c-463d-ae1b-57c2a6ca0d88 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:53.605718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:53.606546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.606956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.607472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.608482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:08:53.608808 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:53.612106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:53.640134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1106/2257] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:08:53.645363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:53.659889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a780-eee4-4dec-92d4-f758f8d24bba None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:53.668111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:53.669202 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.669493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.669869 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.670860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:53.673594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:53.788064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1152/2258] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:08:53 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:53.806511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:53.817273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f5a49-3b7b-402b-a722-b22500007b05 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:53.899586 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:53.900250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:53.900536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:53.900807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:53.901443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:53.903399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:54.049303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1107/2259] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:08:53 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:08:54.050527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:54.062236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e099-d31b-48be-9f58-74dbdbe6f7a1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:54.066945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:54.067551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:54.068031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:54.068392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:54.069045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:54.070961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:54.081368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1153/2260] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:08:54.092784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:54.103891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adffab9-9a21-4c06-907a-651c27725e41 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:54.108670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:54.109268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:54.109552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:54.109824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:54.110467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:54.112217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e7792-703e-46f2-af84-e505fa06d4c3 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:54.124522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1108/2261] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:08:54.135676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:54.149433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee9b6-1d03-4454-8743-a33bde48ccd8 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:54.164216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:54.164216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:54.164216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:54.164216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] PATH_INFO: `/v3/projects/b30d403880184289bcdfac852906872a/users/c9e060b7158340c7b09aa9602059511c/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:54.178429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30d403880184289bcdfac852906872a, user_id=c9e060b7158340c7b09aa9602059511c, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:54.181341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:54.329149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1154/2262] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:08:54 2022] PUT /identity/v3/projects/b30d403880184289bcdfac852906872a/users/c9e060b7158340c7b09aa9602059511c/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:08:54.343774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:54.344564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:54.344564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:54.378374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] MFA Rules not processed for user `c9e060b7158340c7b09aa9602059511c`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:54.380918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:54.450001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:54.481888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1109/2263] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:08:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:08:54.493751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:54.495562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:54.564266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080ec47-d4c1-4c81-9467-d5937e789afc admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:54.569399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:54.575082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:54.575082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:54.575082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:54.580735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:54.594147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761c9c-48e5-476d-afef-dedf6eafe84c service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:54.618176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1155/2264] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:08:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:08:59.461074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:59.573842 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb834d-9781-4287-9da0-90efecee6d35 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:59.607023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:59.609909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:59.609909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:59.609909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] PATH_INFO: `/v3/users/c9e060b7158340c7b09aa9602059511c` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:59.610862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e060b7158340c7b09aa9602059511c)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:59.613104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:08:59.718351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:59.719507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:59.719974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:59.757260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:08:59.759871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:59.794854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:08:59.851307 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1156/2265] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:08:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:08:59.870402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:08:59.870402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:08:59.946277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e060b7158340c7b09aa9602059511c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:59.946956 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e060b7158340c7b09aa9602059511c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:59.955502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e060b7158340c7b09aa9602059511c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:08:59.971543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a1a80-26bd-417f-b306-ef9a19c1d35d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:08:59.979241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:08:59.979858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:08:59.980139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:08:59.980443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:08:59.990163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:08:59.991840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80276e-dcd7-4630-b3f3-e286cddcb384 service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:00.005527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1157/2266] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:08:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:09:00.026894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e060b7158340c7b09aa9602059511c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:00.027714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e060b7158340c7b09aa9602059511c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:00.051589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1110/2267] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:08:59 2022] DELETE /identity/v3/users/c9e060b7158340c7b09aa9602059511c => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:00.056982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:00.057821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:00.058120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:00.079365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:00.080858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:00.152762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1158/2268] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:00.163701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:00.165303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:00.241496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62213467-8137-49bd-9425-0d897f8109b9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:00.247620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:00.248811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:00.249267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:00.249829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:00.257786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:00.264313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:00.265345 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b52ce-84fc-4ce7-8f67-19dff5472941 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:00.277404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1111/2269] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:09:01.093273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:01.099359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:01.117062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487a76a-faf2-4a66-9071-db24cb39a744 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:01.125188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:01.125917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:01.126306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:01.126958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] PATH_INFO: `/v3/projects/b30d403880184289bcdfac852906872a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:01.135640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30d403880184289bcdfac852906872a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:01.135640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:01.339341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:01.343758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30d403880184289bcdfac852906872a', 'request_id': 'req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:01.354757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30d403880184289bcdfac852906872a', 'request_id': 'req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:01.373377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30d403880184289bcdfac852906872a', 'request_id': 'req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:01.495953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] The token cache is being invalidate because project b30d403880184289bcdfac852906872a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:01.498617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:01.498617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-808eabbf-ba33-4bc1-adeb-d82d50a8ed17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:01.501660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1159/2270] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:01 2022] DELETE /identity/v3/projects/b30d403880184289bcdfac852906872a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:01.509248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:01.510256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:01.510761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:01.548223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:01.551204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:01.712512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1112/2271] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:01.720834 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:01.722364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:01.722364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:01.762186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:01.765469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:01.787990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1160/2272] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:01.792508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:01.806508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cad1d-8d29-4772-ae90-c47a4b3bbf95 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:01.818834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:01.825670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:01.844297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1113/2273] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:09:01.845432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:01.866487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923a117-a1cd-4235-bb16-5ffcb90089b7 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:01.883627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:01.884290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:01.884574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:01.884848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:01.885494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:01.887414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:02.190313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1161/2274] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:09:01 2022] POST /identity/v3/projects => generated 369 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:09:02.195062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:02.250677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf891-b0ae-4b71-925f-33b23ed5e297 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:02.351168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:02.351807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:02.353526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:02.353526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:02.353526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:02.357716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:02.611873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:02.613893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:02.636477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623089e0-e56f-4fe5-a4b5-f416fc1bddb3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:02.641570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:02.642664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:02.642664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:02.642868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] PATH_INFO: `/v3/users/9a384e48c8674dd1b926bd1072d65764` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:02.643890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a384e48c8674dd1b926bd1072d65764)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:02.645996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:02.835265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1114/2275] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:09:02 2022] POST /identity/v3/users => generated 285 bytes in 632 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:09:02.836557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:02.847564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fc32-300d-4a98-988f-280f5ae82371 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:02.852492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:02.853105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:02.853383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:02.853652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:02.854409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:02.856333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:02.892052 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1115/2276] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:02.893190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:02.906266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c17982-1d2f-47a5-95e0-5db876aab223 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:02.911669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:02.912646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:02.912934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:02.913207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:02.913866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:02.915516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:02.929668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1116/2277] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:02.939794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:02.951313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b818-8ca2-437c-892c-741026e6f793 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:02.956178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:02.956737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:02.958559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:02.958559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] PATH_INFO: `/v3/projects/3bb77164622c4107a88e1a215b19fbae/users/c149494f33bb472fae0904ec83c8039c/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:02.974810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb77164622c4107a88e1a215b19fbae, user_id=c149494f33bb472fae0904ec83c8039c, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:02.977708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f449-d1af-4ce2-9e35-d0f0372bec4b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:03.055919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a384e48c8674dd1b926bd1072d65764'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:03.056979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a384e48c8674dd1b926bd1072d65764'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:03.067210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a384e48c8674dd1b926bd1072d65764'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:03.209890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1117/2278] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:02 2022] PUT /identity/v3/projects/3bb77164622c4107a88e1a215b19fbae/users/c149494f33bb472fae0904ec83c8039c/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:03.216005 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:03.216321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:03.216598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:03.243478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a384e48c8674dd1b926bd1072d65764'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:03.244675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-62522b17-e7eb-41be-9fec-8de06cb7a666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a384e48c8674dd1b926bd1072d65764'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:03.257912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] MFA Rules not processed for user `c149494f33bb472fae0904ec83c8039c`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:03.259280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:03.279824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:03.289499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1162/2279] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:09:02 2022] DELETE /identity/v3/users/9a384e48c8674dd1b926bd1072d65764 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:03.296955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:03.299360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:03.299588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:03.343244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:03.346484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:03.404116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1118/2280] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:03.482744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1163/2281] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:03.494315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:03.495777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:03.655018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3c86-48c2-42b8-b972-edec1a27c30a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:03.757308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:03.757853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:03.758156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:03.758435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:03.783146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:03.784807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:03.804905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1119/2282] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:04.791651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:04.795646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:04.811125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e33df-f831-431e-af2b-6abfdb41243f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:04.818314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:04.818989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:04.819336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:04.819679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] PATH_INFO: `/v3/projects/98a95555525549a381076118b5431f19` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:04.820848 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a95555525549a381076118b5431f19)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:04.822735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:04.842623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:05.004177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a95555525549a381076118b5431f19', 'request_id': 'req-a7ab888d-eb44-42dc-81bf-01a53999101f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:05.022720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a95555525549a381076118b5431f19', 'request_id': 'req-a7ab888d-eb44-42dc-81bf-01a53999101f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:05.049142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a95555525549a381076118b5431f19', 'request_id': 'req-a7ab888d-eb44-42dc-81bf-01a53999101f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:05.077554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.078785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:05.200031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] The token cache is being invalidate because project 98a95555525549a381076118b5431f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:05.200897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:05.201572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a7ab888d-eb44-42dc-81bf-01a53999101f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:05.203919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1164/2283] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:04 2022] DELETE /identity/v3/projects/98a95555525549a381076118b5431f19 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:05.215322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0b586-09f2-4ed5-acc4-0b4180c1c92b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.215611 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0b586-09f2-4ed5-acc4-0b4180c1c92b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.215952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0b586-09f2-4ed5-acc4-0b4180c1c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.222999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8437f-fd18-43ac-9f71-f401c1f0079f service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.260696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-76d0b586-09f2-4ed5-acc4-0b4180c1c92b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:05.260870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:05.260968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-76d0b586-09f2-4ed5-acc4-0b4180c1c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:05.310845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.314480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.314771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.323403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.323403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:05.351238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1165/2284] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:05.360279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.360726 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.361102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.385916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:05.387142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:05.396497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1166/2285] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:05.400722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.410782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7694-2fed-4588-afc0-fb5b076937b2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.412046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.413633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.420756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.421383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.421664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.422011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.422598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:09:05.422907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.424207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1120/2286] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:09:05.425993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.441330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1167/2287] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:09:05.452066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.469897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cb06a-ac9d-4135-9804-63a6bd09b2e7 service glance] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.477654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.478791 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.479420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.479924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.480659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.482616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.597762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1121/2288] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:09:05 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:09:05.601854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.621513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fa-6e59-48eb-b6e3-b04600587766 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.704477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.705206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.705583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.705951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.706895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.712285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.842687 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1168/2289] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:09:05 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:09:05.844481 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.857394 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689b7f-7700-47f5-b2e8-f9fdbd54ecac admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.862269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.862912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.863195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.863472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.864654 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.866511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.883352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1122/2290] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:05.884639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.895318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641bd3-fd00-413c-9f2b-a2473be8ea41 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.903311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.904111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.904528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.904924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.905676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.907420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27dc49-d022-4fc4-8521-ab6d2887c188 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:05.935279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1169/2291] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:05.936695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:05.954872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc7980-ec20-463e-8ac9-c17883275e9b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:05.961442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:05.962119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:05.962531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:05.962816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] PATH_INFO: `/v3/projects/c0b21ed97aee4e01b4c54ee031cbb523/users/ddca0cdc82e048a8b967157481fc98c5/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:05.986113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b21ed97aee4e01b4c54ee031cbb523, user_id=ddca0cdc82e048a8b967157481fc98c5, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:05.988960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:06.115223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1123/2292] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:05 2022] PUT /identity/v3/projects/c0b21ed97aee4e01b4c54ee031cbb523/users/ddca0cdc82e048a8b967157481fc98c5/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:06.115850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.115850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.116015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.126832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:06.154368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] MFA Rules not processed for user `ddca0cdc82e048a8b967157481fc98c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:06.155781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:06.259696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1170/2293] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:06.260764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:06.414434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5989ed-b261-4d91-8648-b8fb8c980fcc admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:06.419649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:06.420296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.420578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.420854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.421503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:06.423411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:06.531277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1124/2294] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:09:06 2022] POST /identity/v3/projects => generated 377 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:09:06.537808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:06.551659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7c089-fa06-4d36-ac76-7b9adcaf442a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:06.683805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:06.684553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.684937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.685361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.686231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:06.688511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:06.852194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1171/2295] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:09:06 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:09:06.854375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:06.870918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b3e4-a432-4083-918b-db1568c9b856 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:06.876642 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:06.877526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.877901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.878356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.879201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:06.882978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:06.890075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:06.897296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1125/2296] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:06.901660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:06.913681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee38fc-f803-4370-b014-3b1f2050e001 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:06.918907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:06.919637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.920026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.924069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.924069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:06.924069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0f3b-ca64-4172-b032-af2a843a736f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:06.939990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1172/2297] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:06.945192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:06.955614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aeebd-c485-4258-bbcd-a7015ae1a5d2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:06.960495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:06.961295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:06.961578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:06.961848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] PATH_INFO: `/v3/projects/f5ce9590d7a5467fa0ebff98b9740d25/users/4cb04c94b9964247a4548d8a4fcbaa88/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:06.980798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ce9590d7a5467fa0ebff98b9740d25, user_id=4cb04c94b9964247a4548d8a4fcbaa88, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:06.983504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:07.084062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1126/2298] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:06 2022] PUT /identity/v3/projects/f5ce9590d7a5467fa0ebff98b9740d25/users/4cb04c94b9964247a4548d8a4fcbaa88/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:07.094824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82d67e-33e6-438d-bae5-87ca2370bf59 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.095118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82d67e-33e6-438d-bae5-87ca2370bf59 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.095718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82d67e-33e6-438d-bae5-87ca2370bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.128582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-5a82d67e-33e6-438d-bae5-87ca2370bf59 None None] MFA Rules not processed for user `4cb04c94b9964247a4548d8a4fcbaa88`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:07.130641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5a82d67e-33e6-438d-bae5-87ca2370bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:07.230655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:07.235273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1173/2299] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:07.240613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:07.338853 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781305de-320b-47f2-94bb-f38365df647b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:07.343738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:07.345061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.345452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.345805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.346543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:07.349075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:07.418402 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.423425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.423425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.443993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:07.446439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:07.454302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1127/2300] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:09:07 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:09:07.463755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:07.542006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1174/2301] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:09:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:07.591939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:07.593697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:07.597402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d162-bbbd-48c6-bb77-e8f1591b1540 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:07.605830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:07.606655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.607094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.607546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.608578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:07.611053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:07.706440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1128/2302] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:09:07 2022] POST /identity/v3/users => generated 295 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:09:07.711182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:07.745333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1a97-d4aa-4833-ba9a-290fe5a5f652 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:07.764757 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e07cc-c25c-421d-8f70-1c375bfbbdc3 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:07.791653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:07.792456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.792569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.792840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.795527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:07.816305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:07.832193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1129/2303] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:09:07.834301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:07.851829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ef73-a94f-4bc5-9191-f5025863fd46 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:07.889644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:07.890330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.890611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.890882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] PATH_INFO: `/v3/projects/5ba448847df84e24bca7b0f39440e1af/users/2501e55f9dc3413b8f5c6c855f5167eb/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.894409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:07.897750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:07.898246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:07.898682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:07.971257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba448847df84e24bca7b0f39440e1af, user_id=2501e55f9dc3413b8f5c6c855f5167eb, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:07.975271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:08.024949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:08.058607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:08.058607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:08.058607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1175/2304] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:09:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 25 03:09:08.557787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1130/2305] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:07 2022] PUT /identity/v3/projects/5ba448847df84e24bca7b0f39440e1af/users/2501e55f9dc3413b8f5c6c855f5167eb/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:08.558394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:08.565736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f35ad-c730-46cf-a166-1d4b83ce7b8a service swift] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:08.631642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:08.632309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:08.632591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:08.632861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:08.633513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:08.635100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0280fe2-1741-40b0-8079-a23c7d32e64c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:08.650595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1176/2306] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:08.652006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:08.663751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c953f98-48b1-482f-9d0e-abc82096b002 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:08.668889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:08.669630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:08.669983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:08.670356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] PATH_INFO: `/v3/projects/5ba448847df84e24bca7b0f39440e1af/users/2501e55f9dc3413b8f5c6c855f5167eb/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:08.672454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba448847df84e24bca7b0f39440e1af, user_id=2501e55f9dc3413b8f5c6c855f5167eb, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:08.675190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:08.687673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1131/2307] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:08 2022] PUT /identity/v3/projects/5ba448847df84e24bca7b0f39440e1af/users/2501e55f9dc3413b8f5c6c855f5167eb/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:08.703866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:08.704183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:08.704469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:08.733014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] MFA Rules not processed for user `2501e55f9dc3413b8f5c6c855f5167eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:08.734148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:08.806670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1177/2308] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:08.820476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:08.822420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:08.828402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:08.885118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4abd16-9d86-4c71-a8c5-01d02dc75e3b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:08.888813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:08.889473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:08.889767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:08.890054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:08.956210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:08.958892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:08.967800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1132/2309] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:11.849929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:11.849929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:11.864835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8437c-4c15-448b-88a1-7fe372cbb82d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:11.869266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:11.870066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:11.870420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:11.870766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:11.956650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:11.959428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:11.982732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1178/2310] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:13.936744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:13.938459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:14.042914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b73b57-cffc-4c91-9b88-9b13f26593e0 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:14.048425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:14.057448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:14.057448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:14.057448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] PATH_INFO: `/v3/users/1f2a411b051147e58a94dfa03413539b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:14.057448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2a411b051147e58a94dfa03413539b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:14.060338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:14.526582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2a411b051147e58a94dfa03413539b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:14.527240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2a411b051147e58a94dfa03413539b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:14.534784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2a411b051147e58a94dfa03413539b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:14.542945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:14.768290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2a411b051147e58a94dfa03413539b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:14.768935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2a411b051147e58a94dfa03413539b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:14.787502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1133/2311] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:09:13 2022] DELETE /identity/v3/users/1f2a411b051147e58a94dfa03413539b => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:15.048483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:15.057785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3151c4-0665-471a-a606-f962d14dce3c service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:15.063290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:15.111667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:15.112330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.112789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.113139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.115430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.131618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:15.135656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.190818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:15.211175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:15.211175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1179/2312] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:15.211912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2095-b794-47e3-b508-ba0879c53f78 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:15.218242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:15.218880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.219165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.219437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] PATH_INFO: `/v3/projects/d1f677a8a80e4591ab642d64f4008c00` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.220598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f677a8a80e4591ab642d64f4008c00)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:15.222517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:15.568992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f677a8a80e4591ab642d64f4008c00', 'request_id': 'req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:15.584287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f677a8a80e4591ab642d64f4008c00', 'request_id': 'req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:15.704907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:15.706320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.722248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f677a8a80e4591ab642d64f4008c00', 'request_id': 'req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:15.737398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] The token cache is being invalidate because project d1f677a8a80e4591ab642d64f4008c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:15.738357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:15.738849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-faaa0a4f-9c88-485a-bf75-dda9e2a5ef97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:15.744909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1134/2313] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:15 2022] DELETE /identity/v3/projects/d1f677a8a80e4591ab642d64f4008c00 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:15.746401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c50b85-54fd-49ca-9b74-dfce7cdbfd8b None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.746792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c50b85-54fd-49ca-9b74-dfce7cdbfd8b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.747171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c50b85-54fd-49ca-9b74-dfce7cdbfd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.768053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f5c50b85-54fd-49ca-9b74-dfce7cdbfd8b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:15.769500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f5c50b85-54fd-49ca-9b74-dfce7cdbfd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.793366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e619fb-3984-4f66-ba55-a1944d2d1ce1 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:15.843350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1135/2314] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:15.851746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.857476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.857476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.864957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:15.870299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:15.872075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:15.872813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.873230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.874172 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.876225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.878752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:15.880676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:15.893803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:15.894679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1136/2315] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:15.896289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:15.898639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:15.907672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579875f3-8819-45e8-b366-9e7eff969875 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:15.908415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1180/2316] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:09:15.917467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:15.918123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.918404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.918676 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.919362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:09:15.919674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:15.921500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:15.935079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1137/2317] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:09:15.939770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:15.951247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134ab05-53be-4b2e-b781-2d46cff65fc1 service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:15.957038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:15.958014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:15.958371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:15.958783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:15.959675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:15.961875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.058003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1181/2318] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:09:15 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:09:16.062297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:16.074415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f369b93-0adc-4711-a3e1-817ad5bf2b5e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:16.145619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:16.146527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.146807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.147143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.147849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:16.150155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.233234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1138/2319] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:09:16 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:09:16.238197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:16.250364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d016d1-fe2f-4d10-bae5-caef22ff40de admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:16.255749 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:16.256443 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.256728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.257002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.257686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:16.259363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.270732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1182/2320] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:16.275029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:16.285283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ebd93-3ce4-4c1d-9754-f72bfef77c78 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:16.290801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:16.291412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.291709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.292014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.292701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:16.294338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563ed59-c2a4-4e42-bd55-c0721f380d1f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.305358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1139/2321] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:09:16.309537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:16.320280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a7e3-422e-4eb5-9362-a490d3a77f6f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:16.328932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:16.328932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.328932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.328932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] PATH_INFO: `/v3/projects/bca65195051c44f5b5af05e61295841f/users/9994bba5874c4834bee1b83cbc9a2dea/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.343994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca65195051c44f5b5af05e61295841f, user_id=9994bba5874c4834bee1b83cbc9a2dea, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:16.346669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.461522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1183/2322] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:09:16 2022] PUT /identity/v3/projects/bca65195051c44f5b5af05e61295841f/users/9994bba5874c4834bee1b83cbc9a2dea/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:16.474244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d671414-0429-475f-b377-6cee106342bb None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.474556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d671414-0429-475f-b377-6cee106342bb None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.475062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d671414-0429-475f-b377-6cee106342bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.516707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5d671414-0429-475f-b377-6cee106342bb None None] MFA Rules not processed for user `9994bba5874c4834bee1b83cbc9a2dea`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:16.518214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5d671414-0429-475f-b377-6cee106342bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:16.593502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1140/2323] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:16.605342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:16.607045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:16.670197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214381ad-d9ee-45c1-82ce-4cc2fe8e38f2 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:16.674103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:16.674760 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:16.675044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:16.675341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:16.684894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:16.686378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb9b6b-9b5e-4f45-ae2a-043ca58dfb9e service glance] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:16.695827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1184/2324] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:20.529594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d671414-0429-475f-b377-6cee106342bb None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:20.531136 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5d671414-0429-475f-b377-6cee106342bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:20.622861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:20.630438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d671414-0429-475f-b377-6cee106342bb None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:20.642041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:20.642041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:20.642041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:20.642041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] PATH_INFO: `/v3/users/ddca0cdc82e048a8b967157481fc98c5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:20.642503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddca0cdc82e048a8b967157481fc98c5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:20.642503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:21.035506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddca0cdc82e048a8b967157481fc98c5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:21.036835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddca0cdc82e048a8b967157481fc98c5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:21.044268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddca0cdc82e048a8b967157481fc98c5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:21.179693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddca0cdc82e048a8b967157481fc98c5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:21.180557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddca0cdc82e048a8b967157481fc98c5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:21.201963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1141/2325] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:09:20 2022] DELETE /identity/v3/users/ddca0cdc82e048a8b967157481fc98c5 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:21.206510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161f4d-a736-4ce6-8021-775b3ea63e52 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:21.206994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161f4d-a736-4ce6-8021-775b3ea63e52 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:21.207088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161f4d-a736-4ce6-8021-775b3ea63e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:21.226080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f8161f4d-a736-4ce6-8021-775b3ea63e52 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:21.227278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f8161f4d-a736-4ce6-8021-775b3ea63e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:21.287954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1185/2326] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:21.298603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:21.300040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:21.378155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29838f3-56bc-46cb-a5d4-1014f5072281 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:21.383491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:21.384330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:21.384538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:21.384951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:21.400506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:21.402823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5adf8-aa6e-4c17-883f-e3018acaf094 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:21.418480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1142/2327] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:21.672786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:21.673976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:21.674729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:21.677207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1186/2328] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:09:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:09:21.683342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:21.683867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:21.683867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:21.706142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:21.708521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:21.804241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1143/2329] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:09:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:21.947873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:21.949585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:21.990821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230035-8f98-408b-b9c5-20aae33d64a1 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:22.006544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:22.007214 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:22.007498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:22.007773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] PATH_INFO: `/v3/projects/c0b21ed97aee4e01b4c54ee031cbb523` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:22.009029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b21ed97aee4e01b4c54ee031cbb523)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:22.010922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:22.118341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:22.132942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b21ed97aee4e01b4c54ee031cbb523', 'request_id': 'req-f7aade82-c8e2-48f5-9482-20bc0d98deba'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.139163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b21ed97aee4e01b4c54ee031cbb523', 'request_id': 'req-f7aade82-c8e2-48f5-9482-20bc0d98deba'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.145755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b21ed97aee4e01b4c54ee031cbb523', 'request_id': 'req-f7aade82-c8e2-48f5-9482-20bc0d98deba'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.252405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] The token cache is being invalidate because project c0b21ed97aee4e01b4c54ee031cbb523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:22.253835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.255796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.261589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1187/2330] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:21 2022] DELETE /identity/v3/projects/c0b21ed97aee4e01b4c54ee031cbb523 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:22.262026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:22.275240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb826bb3-52e1-4f3a-91fe-4aa3cc05a2ff None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:22.281858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:22.346158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:22.346807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:22.347091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:22.347362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] PATH_INFO: `/v3/users/4cb04c94b9964247a4548d8a4fcbaa88` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:22.348631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb04c94b9964247a4548d8a4fcbaa88)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:22.351037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:22.622570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb04c94b9964247a4548d8a4fcbaa88'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.623230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb04c94b9964247a4548d8a4fcbaa88'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.630876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb04c94b9964247a4548d8a4fcbaa88'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.733742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb04c94b9964247a4548d8a4fcbaa88'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.734438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb04c94b9964247a4548d8a4fcbaa88'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:22.753743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1144/2331] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:09:22 2022] DELETE /identity/v3/users/4cb04c94b9964247a4548d8a4fcbaa88 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:23.221100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:23.241130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:23.280747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aade82-c8e2-48f5-9482-20bc0d98deba admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:23.330143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:23.330754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.331035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.331310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.333694 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:23.352572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:23.357565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:23.434842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:23.436371 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:23.437628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b2064-6a70-4d14-bc60-817c2c91dddb admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:23.444203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:23.444847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.445208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.445576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] PATH_INFO: `/v3/projects/f5ce9590d7a5467fa0ebff98b9740d25` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.446843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ce9590d7a5467fa0ebff98b9740d25)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:23.449069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:23.452239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1188/2332] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:23.620657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ce9590d7a5467fa0ebff98b9740d25', 'request_id': 'req-ad96f183-cc34-4832-bd5f-b927d1164856'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:23.626116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ce9590d7a5467fa0ebff98b9740d25', 'request_id': 'req-ad96f183-cc34-4832-bd5f-b927d1164856'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:23.710451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ce9590d7a5467fa0ebff98b9740d25', 'request_id': 'req-ad96f183-cc34-4832-bd5f-b927d1164856'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:23.722634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] The token cache is being invalidate because project f5ce9590d7a5467fa0ebff98b9740d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:23.722978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:23.723613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-ad96f183-cc34-4832-bd5f-b927d1164856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:23.725265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1145/2333] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:23 2022] DELETE /identity/v3/projects/f5ce9590d7a5467fa0ebff98b9740d25 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:23.729649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:23.741444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7916e-5cb4-4327-b8a0-7a5248b16eb3 service glance] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:23.796943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:23.797598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.797881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.798157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] PATH_INFO: `/v3/users/2501e55f9dc3413b8f5c6c855f5167eb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.799298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2501e55f9dc3413b8f5c6c855f5167eb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:23.801940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:23.815767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33c5b-d5e4-47b6-b8a4-5ba96f6b721a None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.816414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33c5b-d5e4-47b6-b8a4-5ba96f6b721a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.816414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd33c5b-d5e4-47b6-b8a4-5ba96f6b721a None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.817848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1146/2334] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:09:23.822514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.822890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.823174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.843513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:23.844792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:23.896849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1147/2335] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:09:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:23.908712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:23.910157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:23.920800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52608-7133-430e-8c1e-86e1ada5903f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:23.924655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:23.925261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:23.925544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:23.925820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:23.933962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:23.935434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c893559-62a5-4dee-93bc-32e5ffa3634f service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:23.944303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1148/2336] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:09:24.032396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59825d70-29b9-43fd-bda0-8dad3518bcb9 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:24.032747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59825d70-29b9-43fd-bda0-8dad3518bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:24.033030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59825d70-29b9-43fd-bda0-8dad3518bcb9 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:24.034361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1149/2337] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:09:24.040785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:24.041074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:24.041356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:24.063163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:24.064434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:24.096614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2501e55f9dc3413b8f5c6c855f5167eb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:24.097405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2501e55f9dc3413b8f5c6c855f5167eb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:24.107302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:24.120366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2501e55f9dc3413b8f5c6c855f5167eb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:24.141630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1150/2338] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:09:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:09:24.152513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:24.154017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:24.166832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adae4c-1776-4e5b-b8ca-47b079c21434 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:24.171247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:24.171913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:24.172248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:24.176210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:24.184278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:24.185779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:24.197781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1151/2339] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:09:24.199545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2501e55f9dc3413b8f5c6c855f5167eb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:24.200286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-09d4bb11-55cb-41ee-860e-fdb17fef2733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2501e55f9dc3413b8f5c6c855f5167eb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:24.219617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1189/2340] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:09:23 2022] DELETE /identity/v3/users/2501e55f9dc3413b8f5c6c855f5167eb => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:09:24.471082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:24.976524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:24.978037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:24.993621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aa250-377f-4dc3-928c-217ea1db23bc service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:25.052722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:25.053267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:25.053546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:25.053816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] PATH_INFO: `/v3/projects/5ba448847df84e24bca7b0f39440e1af` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:25.054918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba448847df84e24bca7b0f39440e1af)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:25.056874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:25.167788 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba448847df84e24bca7b0f39440e1af', 'request_id': 'req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:25.173920 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba448847df84e24bca7b0f39440e1af', 'request_id': 'req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:25.236874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba448847df84e24bca7b0f39440e1af', 'request_id': 'req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:25.249261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] The token cache is being invalidate because project 5ba448847df84e24bca7b0f39440e1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:09:25.249555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:25.250359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:09:25.252027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1152/2341] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:09:24 2022] DELETE /identity/v3/projects/5ba448847df84e24bca7b0f39440e1af => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:09:26.894066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:29.856885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:29.857428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:29.857428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:29.877467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:09:29.878692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:29.938634 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1190/2342] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:09:29.951169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:29.952644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:30.038211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4bb67-2ca9-4e16-8326-189ccb912eb5 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:30.048063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:30.049077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:30.049744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:30.050080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:30.064421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:30.067065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:30.080702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1153/2343] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:09:39.931558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:40.077235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:09:45.350428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:09:45.351849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:45.410783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5b8e-b0a8-4d81-afbc-9564dc4828cc None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:09:45.415636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:09:45.416268 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:09:45.416573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:09:45.416819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:09:45.418625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:09:45.482782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:09:45.484237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a018555-f524-4afb-bf44-0a8f6ddce3c7 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:09:45.494215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1191/2344] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:09:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:09:55.489441 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=50 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:00.200536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:00.202133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:00.267486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777df8-b514-4e76-9c3d-35d9e49e6cd5 service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:00.272054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:00.272710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:00.272993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:00.273269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] PATH_INFO: `/v3/users/c149494f33bb472fae0904ec83c8039c` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:00.274939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c149494f33bb472fae0904ec83c8039c)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:00.276941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:00.587898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c149494f33bb472fae0904ec83c8039c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:00.588590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c149494f33bb472fae0904ec83c8039c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:00.595664 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c149494f33bb472fae0904ec83c8039c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:00.761520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c149494f33bb472fae0904ec83c8039c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:00.762200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c149494f33bb472fae0904ec83c8039c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:00.781895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1154/2345] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:10:00 2022] DELETE /identity/v3/users/c149494f33bb472fae0904ec83c8039c => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:00.786567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:00.787067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:00.787177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:00.807892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:00.809129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:00.869732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1192/2346] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:00.879748 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:00.882395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:00.940480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245da1a-60c0-4a37-8213-63b0400467f8 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:00.944066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:00.944770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:00.945047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:00.945319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:00.954393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:00.955821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a17e6-e7c6-4673-b8f4-400350b4308e service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:00.966119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1155/2347] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:01.426504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:01.427565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:01.442090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e602f-db64-4ba5-9c26-090e50596ff2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:01.447645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:01.448330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:01.449180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:01.449465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] PATH_INFO: `/v3/projects/3bb77164622c4107a88e1a215b19fbae` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:01.450974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb77164622c4107a88e1a215b19fbae)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:01.457073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:01.574234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb77164622c4107a88e1a215b19fbae', 'request_id': 'req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:01.579791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb77164622c4107a88e1a215b19fbae', 'request_id': 'req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:01.586322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb77164622c4107a88e1a215b19fbae', 'request_id': 'req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:01.669858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] The token cache is being invalidate because project 3bb77164622c4107a88e1a215b19fbae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:10:01.670146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:01.670772 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6f07cd9f-16d7-41df-8d4d-b8ac3bb4043a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:01.674466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1193/2348] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:10:01 2022] DELETE /identity/v3/projects/3bb77164622c4107a88e1a215b19fbae => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:01.677254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:01.677702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:01.677794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:01.698990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:01.700318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:01.717396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:01.761406 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1156/2349] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:01.771228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:01.771228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:01.772049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:01.800296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:01.800296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:01.812368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1194/2350] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:01.816949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:01.831604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4025-2989-4de0-a126-96ccdceb0e8f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:01.837035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:01.837959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:01.838247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:01.838589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:01.839233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:10:01.839542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:01.841331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:01.854298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1157/2351] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:10:01.858942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:01.869548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b153-b25f-4543-b332-d04d745dce91 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:01.874682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:01.875329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:01.875605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:01.875870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:01.876547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:01.878771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:01.974729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1195/2352] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:10:01 2022] POST /identity/v3/projects => generated 397 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:10:01.979860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:01.990646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5537f-4c24-4e36-af32-5079d939189a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:02.046929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:02.047458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.047739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.048010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.048667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:02.050590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:02.179741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1158/2353] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:10:01 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:10:02.188247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:02.199879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba58fd-d2a1-4588-9dcf-1566ca025d7e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:02.204867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:02.205471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.205751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.206086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.206735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:02.208368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:02.219570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1196/2354] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:02.223697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:02.236057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7e35a-9baa-489c-99ba-aa430b88749c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:02.241246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:02.241858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.242142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.242417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.243094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:02.244995 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777f692-a76f-4df3-a39b-1b8e5926e912 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:02.263584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1159/2355] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:02.269992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:02.281577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e47a4-8f4a-403f-a108-342583f34f6b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:02.286925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:02.287714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.288000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.288464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] PATH_INFO: `/v3/projects/cf1f2f960c344cacafb2cea8aa59557a/users/503a9ae58b2447a2944604134dc3b5fb/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.306030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1f2f960c344cacafb2cea8aa59557a, user_id=503a9ae58b2447a2944604134dc3b5fb, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:02.313362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:02.431583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1197/2356] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:10:02 2022] PUT /identity/v3/projects/cf1f2f960c344cacafb2cea8aa59557a/users/503a9ae58b2447a2944604134dc3b5fb/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:10:02.443555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.443884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.444238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.478474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] MFA Rules not processed for user `503a9ae58b2447a2944604134dc3b5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:02.480697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:02.558416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1160/2357] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:02.569721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:02.572230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:02.594806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:02.636342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9425e70-b0e3-4d1f-b809-0d2de344323c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:02.640778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:02.641396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:02.641679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:02.641951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:02.651512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:02.652975 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595a1e-80c0-4194-b1a5-1f5f6313d894 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:02.662891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1198/2358] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:02 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:04.215567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:04.217081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:04.237829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:04.282660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363b10a-5888-4376-951d-cdfd2788c257 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:04.287278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:04.287930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:04.288236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:04.288508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] PATH_INFO: `/v3/users/9994bba5874c4834bee1b83cbc9a2dea` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:04.289623 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9994bba5874c4834bee1b83cbc9a2dea)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:04.292501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:04.489123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9994bba5874c4834bee1b83cbc9a2dea'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:04.489718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9994bba5874c4834bee1b83cbc9a2dea'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:04.497249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9994bba5874c4834bee1b83cbc9a2dea'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:04.613116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9994bba5874c4834bee1b83cbc9a2dea'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:04.613735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9994bba5874c4834bee1b83cbc9a2dea'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:04.632125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1161/2359] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:10:04 2022] DELETE /identity/v3/users/9994bba5874c4834bee1b83cbc9a2dea => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:04.636864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:04.637332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:04.637469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:04.656318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:04.657529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:04.720775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1199/2360] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:04.731423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:04.732888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:04.793208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff57cc-91d9-4faa-9b9a-ea952f631546 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:04.797508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:04.798120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:04.798396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:04.798670 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:04.808008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:04.809492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262472d3-974a-4e54-9cde-389f064ca05b service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:04.820344 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1162/2361] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:05.295385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:05.296883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:05.309943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800734a-2408-45fe-a914-37ae0b9ef7e6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:05.314591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:05.315240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:05.315524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:05.316036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] PATH_INFO: `/v3/projects/bca65195051c44f5b5af05e61295841f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:05.316992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca65195051c44f5b5af05e61295841f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:05.318898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:05.494969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca65195051c44f5b5af05e61295841f', 'request_id': 'req-5a2f5eef-df5c-42da-aa20-d12164744211'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:05.501181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca65195051c44f5b5af05e61295841f', 'request_id': 'req-5a2f5eef-df5c-42da-aa20-d12164744211'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:05.508375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca65195051c44f5b5af05e61295841f', 'request_id': 'req-5a2f5eef-df5c-42da-aa20-d12164744211'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:05.742267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] The token cache is being invalidate because project bca65195051c44f5b5af05e61295841f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:10:05.742827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:05.744395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-5a2f5eef-df5c-42da-aa20-d12164744211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:05.748026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1200/2362] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:10:05 2022] DELETE /identity/v3/projects/bca65195051c44f5b5af05e61295841f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:05.754688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:05.755172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:05.755783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:05.777656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:05.778854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:05.812354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:05.838245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1163/2363] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:05.846660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:05.846957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:05.847262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:05.870068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:05.871283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:05.879651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1201/2364] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:05.884266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:05.894310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd5748-08de-422c-8140-0a2b3650b2d3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:05.899042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:05.899929 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:05.900416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:05.900699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:05.901360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:10:05.901674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:05.903271 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:05.919580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1164/2365] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:10:05.920919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:05.931896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d41b71-3e64-4fee-a184-92a42feddad6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:05.937165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:05.937777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:05.938213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:05.938510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:05.939127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:05.941051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.049938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1202/2366] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:10:05 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:10:06.053997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.064388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd96925-9214-4cc9-8a7c-1e0df203d168 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.122574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.123187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.123522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.123800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.124498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.126545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.253045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1165/2367] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:10:06 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:10:06.257497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.268689 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665663-5712-46ba-ae6e-1161041592f9 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.273109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.273704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.273987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.274322 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.274964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.276592 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.287057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1203/2368] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:06.291043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.300954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a68c7a-d54d-4470-a29b-9e18841ba819 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.306992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.307583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.307861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.308177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.308821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.310417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e683-6f81-49c3-8809-711c69c8697a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.320898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1166/2369] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:06.326581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.336375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f2ba-7c6f-4129-9e8d-dd4e6ef1fe09 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.342012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.342667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.342962 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.343262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] PATH_INFO: `/v3/projects/6a026a3acf7a4c6f9bf1da687d18d42b/users/1ee5d51d8e74472d97d4e5bd6c79b069/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.345498 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:06.358714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a026a3acf7a4c6f9bf1da687d18d42b, user_id=1ee5d51d8e74472d97d4e5bd6c79b069, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.362433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.453526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1204/2370] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:10:06 2022] PUT /identity/v3/projects/6a026a3acf7a4c6f9bf1da687d18d42b/users/1ee5d51d8e74472d97d4e5bd6c79b069/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:06.463908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.464232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.464511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.491210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] MFA Rules not processed for user `1ee5d51d8e74472d97d4e5bd6c79b069`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:06.492518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:06.555232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1167/2371] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:10:06.567098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.568803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:06.626142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86cf5c-8815-4757-8618-74c2c4220a10 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.630018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.630613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.630888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.631158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.639511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.640947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f7d01-2969-42a2-a18d-050379c5514b service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:06.649691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1205/2372] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:06.674168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:06.712203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:06.734614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e07a6-bd60-401a-98bc-b44aa1de4278 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:06.739111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:06.739727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:06.740004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:06.740313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] PATH_INFO: `/v3/users/1ee5d51d8e74472d97d4e5bd6c79b069` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:06.741426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee5d51d8e74472d97d4e5bd6c79b069)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:06.744022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:07.119303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee5d51d8e74472d97d4e5bd6c79b069'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:07.119959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee5d51d8e74472d97d4e5bd6c79b069'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:07.127230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee5d51d8e74472d97d4e5bd6c79b069'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:07.264773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee5d51d8e74472d97d4e5bd6c79b069'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:07.265402 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee5d51d8e74472d97d4e5bd6c79b069'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:07.283295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1168/2373] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:10:06 2022] DELETE /identity/v3/users/1ee5d51d8e74472d97d4e5bd6c79b069 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:07.287861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:07.288487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:07.288487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:07.308630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:07.309897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:07.368459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1206/2374] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:07.378539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:07.380010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:07.437738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea9a9-9632-40e4-a0de-9b3f75d61e46 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:07.441502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:07.442106 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:07.442387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:07.442661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:07.451890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:07.458153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c36fd-af67-4bf8-ba17-930302dcf7ac service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:07.462843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1169/2375] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:07.974908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:07.976379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:07.990185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625ee96-5caa-46d1-a1c1-99662bb8e20f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:07.995463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:07.996101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:07.997029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:07.997314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] PATH_INFO: `/v3/projects/6a026a3acf7a4c6f9bf1da687d18d42b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:07.998658 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a026a3acf7a4c6f9bf1da687d18d42b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:08.000588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:08.264089 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a026a3acf7a4c6f9bf1da687d18d42b', 'request_id': 'req-0c10713f-b909-47c0-ace7-4c8ec0963a8f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:08.272128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a026a3acf7a4c6f9bf1da687d18d42b', 'request_id': 'req-0c10713f-b909-47c0-ace7-4c8ec0963a8f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:08.280369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a026a3acf7a4c6f9bf1da687d18d42b', 'request_id': 'req-0c10713f-b909-47c0-ace7-4c8ec0963a8f'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:08.846828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] The token cache is being invalidate because project 6a026a3acf7a4c6f9bf1da687d18d42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:10:08.847435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:08.847718 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c10713f-b909-47c0-ace7-4c8ec0963a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:08.849284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1207/2376] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:10:07 2022] DELETE /identity/v3/projects/6a026a3acf7a4c6f9bf1da687d18d42b => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:08.854825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:08.855277 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:08.855504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:08.876957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:08.878600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:08.931312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:08.940300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1170/2377] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:10:08.949965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:08.950233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:08.950507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:08.970267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:08.971456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:08.979415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1208/2378] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:10:08.986619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:08.996787 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377960-ca57-4140-8b8b-cc9bd2ba8c36 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.001307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.001901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.002178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.002449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.003119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:10:09.003677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.005316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.017713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1171/2379] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:10:09.021964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.027065 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:09.032436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc320-ec54-48ec-8b52-0b660a95101e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.036972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.037570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.037846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.038146 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.038777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.040907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.140215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1209/2380] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:10:09 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:10:09.147204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.155054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea9f36-3bb3-4efc-bda9-aa2131868c7e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.209982 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.210577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.210876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.211152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.211891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.213879 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.333146 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1172/2381] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:10:09 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:10:09.339242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.349306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cb9c8-2b30-4ea4-9391-4a059ee46828 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.354225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.354938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.355094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.355370 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.355988 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.357596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.368589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1210/2382] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:09.372986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.384428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88230d-c552-402e-98a2-c582d90f2623 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.389275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.389976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.390310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.390521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.391163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.392800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420ae7-153a-473f-9953-1e78d49d7908 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.404199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1173/2383] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:10:09.408514 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.419030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5169b1-f8ac-4188-a2e6-b154631a3365 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.423527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.424196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.424477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.424751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] PATH_INFO: `/v3/projects/aa73d487b3884350889e14e0b018e0f1/users/be1baf177c664022ac3e872df3a3cd27/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.440124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa73d487b3884350889e14e0b018e0f1, user_id=be1baf177c664022ac3e872df3a3cd27, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.442782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.561912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1211/2384] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:10:09 2022] PUT /identity/v3/projects/aa73d487b3884350889e14e0b018e0f1/users/be1baf177c664022ac3e872df3a3cd27/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:09.567210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a108f5-70b5-409d-ae11-a3ffa78be887 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.567502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a108f5-70b5-409d-ae11-a3ffa78be887 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.567782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a108f5-70b5-409d-ae11-a3ffa78be887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.588715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-91a108f5-70b5-409d-ae11-a3ffa78be887 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:09.598442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-91a108f5-70b5-409d-ae11-a3ffa78be887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:09.650972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1174/2385] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:09.660433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:09.662285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:09.719299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88309395-7628-4878-86a8-0d5b8c373eda admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:09.722971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:09.723573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:09.723853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:09.724129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:09.733207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:09.734619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:09.743952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1212/2386] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:11.602433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbc927-bbd7-47ec-8e69-eaf06b5fb72d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:11.603443 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbc927-bbd7-47ec-8e69-eaf06b5fb72d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:11.603733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbc927-bbd7-47ec-8e69-eaf06b5fb72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:11.631565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9dbbc927-bbd7-47ec-8e69-eaf06b5fb72d None None] MFA Rules not processed for user `be1baf177c664022ac3e872df3a3cd27`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:11.635198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9dbbc927-bbd7-47ec-8e69-eaf06b5fb72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:11.689677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:11.700261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1175/2387] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:11.711006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:11.713232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:11.769707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385a68-43fd-49e5-83ba-cdb94b9aee3d service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:11.773206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:11.773805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:11.774613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:11.774910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:11.783392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:11.784836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc3294-e824-4e14-aac9-5617989527a5 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:11.794435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1213/2388] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:13.729032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:13.729603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:13.729603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:13.749352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] MFA Rules not processed for user `be1baf177c664022ac3e872df3a3cd27`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:13.750580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:13.757738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1176/2389] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:15.252751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:15.263067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:15.263067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:15.275329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:15.276655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:15.329528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:15.342368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1214/2390] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:10:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:15.365731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:15.367346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:15.445533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e45be-64d1-45d5-a280-900e03488e03 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:15.450679 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:15.451352 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:15.451694 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:15.452036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:15.464232 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:15.467252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add151b4-708d-4040-836c-dc64f76cc3ca service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:15.476464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1177/2391] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:10:15.661893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:15.663310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:15.678349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4d36-51f1-4f43-b3e3-9307a9100621 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:15.683631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:15.684353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:15.684635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:15.684911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] PATH_INFO: `/v3/users/503a9ae58b2447a2944604134dc3b5fb` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:15.686063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=503a9ae58b2447a2944604134dc3b5fb)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:15.688070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:15.974088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503a9ae58b2447a2944604134dc3b5fb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:15.974088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503a9ae58b2447a2944604134dc3b5fb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:15.980001 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503a9ae58b2447a2944604134dc3b5fb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:16.052746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503a9ae58b2447a2944604134dc3b5fb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:16.053381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503a9ae58b2447a2944604134dc3b5fb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:16.071867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1215/2392] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:10:15 2022] DELETE /identity/v3/users/503a9ae58b2447a2944604134dc3b5fb => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:16.076464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:16.076810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:16.077221 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:16.097825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:16.099092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:16.162948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1178/2393] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:10:16.170649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:16.172216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:16.230845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b80e-8aed-466d-ab0f-1787cbf98c1a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:16.234414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:16.235021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:16.235302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:16.235574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:16.244773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:16.246206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b616721-f399-41c6-b94e-3c4ee0d7a25d service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:16.257037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1216/2394] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:10:16.877557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:10:16.879235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:16.893469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6003928-db58-463a-8369-05a5b0d24fc1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:10:16.898627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:10:16.900100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:16.900405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:16.900809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] PATH_INFO: `/v3/projects/cf1f2f960c344cacafb2cea8aa59557a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:16.902545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1f2f960c344cacafb2cea8aa59557a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:10:16.904957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:10:17.061087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1f2f960c344cacafb2cea8aa59557a', 'request_id': 'req-0085b3d3-4bc2-4d06-9993-d54680b99807'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:17.066561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1f2f960c344cacafb2cea8aa59557a', 'request_id': 'req-0085b3d3-4bc2-4d06-9993-d54680b99807'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:17.156441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1f2f960c344cacafb2cea8aa59557a', 'request_id': 'req-0085b3d3-4bc2-4d06-9993-d54680b99807'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:17.165507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:17.170181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] The token cache is being invalidate because project cf1f2f960c344cacafb2cea8aa59557a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:10:17.170488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:17.171143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:10:17.172673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1179/2395] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:10:16 2022] DELETE /identity/v3/projects/cf1f2f960c344cacafb2cea8aa59557a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:10:26.251497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:10:53.391815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:10:53.392490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:10:53.392490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:10:53.410853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] MFA Rules not processed for user `7daac509316048ec903a5d78d3586aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:10:53.412099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:10:53.465511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1217/2396] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:10:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:03.458996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=25 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:05.017011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:05.018489 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:05.076648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b3d3-4bc2-4d06-9993-d54680b99807 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:05.080807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:05.081415 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:05.081845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:05.081945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:05.083673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:05.155736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:05.157226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:05.168045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1180/2397] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:06.353704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:06.355164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:06.367904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16fb1-420e-46b9-b46e-c164545e6d12 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:06.371553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:06.372173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:06.372491 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:06.372770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:06.374453 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:06.388779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:06.390203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b61b-5890-4901-991f-bd812f8a903d service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:06.401646 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1218/2398] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:12.233778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:12.235803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:12.250863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf535db0-587e-40e0-b288-b65ffeeeb9cd service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:12.257420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:12.258244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:12.258526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:12.258804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:12.260641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:12.322372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:12.323998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:12.333270 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1181/2399] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:14.900394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1f24-a5c7-45c5-ab2a-dc64b09d4dbb None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:14.900937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1f24-a5c7-45c5-ab2a-dc64b09d4dbb None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:14.900937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec1f24-a5c7-45c5-ab2a-dc64b09d4dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:14.920992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-56ec1f24-a5c7-45c5-ab2a-dc64b09d4dbb None None] MFA Rules not processed for user `fe93db6503a143ab9ed50fedbc8f010f`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:14.922292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-56ec1f24-a5c7-45c5-ab2a-dc64b09d4dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:14.930504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1219/2400] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:11:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:22.329747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=82 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:22.672581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:22.684363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c782ee2-8461-40bb-86ee-0769cf75b94b service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:22.690447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:22.691116 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:22.691463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:22.691803 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:22.693738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:22.753976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:22.755576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:22.764619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1182/2401] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:24.919836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=9 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:27.818890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:27.819478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:27.819478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:27.839180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] MFA Rules not processed for user `be1baf177c664022ac3e872df3a3cd27`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:27.840410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:27.898729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1220/2402] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:11:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:28.132954 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:28.142331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103e74-d6c5-4381-a9d8-f7cc594b665a service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:28.146259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:28.147087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:28.147440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:28.147722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:28.149529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:28.158940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:28.160405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:28.169167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1183/2403] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:28.545393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:28.546732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:28.642509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77281f-e93b-450e-8d0f-4947b94ec723 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:28.653675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:28.654377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:28.654657 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:28.654930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:28.656755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:28.670688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:28.672341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604507-29dc-4a7b-8d45-e49541061b77 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:28.682777 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1221/2404] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:31.614923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:31.633468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f9409-6889-4d97-b8a0-3120be46475d service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:31.638131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:31.638976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:31.642733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:31.642733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:31.642733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:31.655947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:31.657510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:31.667293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1184/2405] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:11:32.261325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:32.261818 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:32.261927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:32.282252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] MFA Rules not processed for user `804df9333a474723968f97eb9d6bfc69`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:11:32.283472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:32.291729 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1222/2406] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:11:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:11:36.194054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:36.195536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:36.219912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a54ce-51d4-407a-a63c-24253ed7683b service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:36.226139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:36.226857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:36.227019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:36.227390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:36.234782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:36.249566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:36.251528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:36.264265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1185/2407] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:11:38.138304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:38.152056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4365f-e109-4af9-8b3e-8ef0a3cdbfc6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:38.157626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:38.158237 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:38.158519 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:38.158799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:38.160561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:38.173860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:38.175666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:38.188253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1223/2408] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:11:42.084436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:11:42.085844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:42.142531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255271b0-3f9d-4e65-abba-5aba85f7887f service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:11:42.146681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:11:42.147274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:11:42.147548 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:11:42.147819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:11:42.149635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:11:42.160574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:11:42.162255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d102a-748c-4241-bc2f-021a387a71da service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:11:42.171338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1186/2409] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:11:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:11:48.182188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=66 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:11:52.167122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=72 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:19.240057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:19.249661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53390-1d49-4da7-8b5d-d9bc4a6c6d33 service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:19.253579 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:19.258545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:19.258545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:19.258545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:19.258545 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:19.319800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:19.321277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fa567-c852-44b1-8c21-2036c697362d service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:19.330963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1224/2410] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:12:23.612796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:23.613308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:23.614087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:23.615428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1187/2411] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:23.649661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:23.650213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:23.650356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:23.672720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] MFA Rules not processed for user `8912c5edc2964df1afb96fd23f02e299`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:23.673959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:23.683536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1225/2412] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:12:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:23.695057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:23.696794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:23.713551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec50a-ca5d-4630-b848-56a75aef679b None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:23.718728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:23.719321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:23.719598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:23.719868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:23.728614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:23.730194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:23.740782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1188/2413] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:23.756572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:23.757969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:23.773656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4a71d-8424-4e67-a862-19af5ed61180 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:23.778347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:23.778981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:23.779260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:23.779586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] PATH_INFO: `/v3/users/7daac509316048ec903a5d78d3586aad` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:23.780697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7daac509316048ec903a5d78d3586aad)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:23.782578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:24.025232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7daac509316048ec903a5d78d3586aad'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.025802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7daac509316048ec903a5d78d3586aad'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.032508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7daac509316048ec903a5d78d3586aad'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.105143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7daac509316048ec903a5d78d3586aad'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.105752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-4b1b9207-9123-4e79-9569-ccb66055b9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7daac509316048ec903a5d78d3586aad'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.122833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1226/2414] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:12:23 2022] DELETE /identity/v3/users/7daac509316048ec903a5d78d3586aad => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:12:24.653850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:24.671745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c329f8-df4a-45ac-84cf-57e78583a36d service placement] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:24.733414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:24.734111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:24.734395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:24.734669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] PATH_INFO: `/v3/projects/d5a66dfea4c5492da1ac3e1f76cbe35d` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:24.735870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a66dfea4c5492da1ac3e1f76cbe35d)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:24.738123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:24.842283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a66dfea4c5492da1ac3e1f76cbe35d', 'request_id': 'req-cfa185a0-68ee-4080-9337-e4696a92bec2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.848014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a66dfea4c5492da1ac3e1f76cbe35d', 'request_id': 'req-cfa185a0-68ee-4080-9337-e4696a92bec2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.948719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a66dfea4c5492da1ac3e1f76cbe35d', 'request_id': 'req-cfa185a0-68ee-4080-9337-e4696a92bec2'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.961543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] The token cache is being invalidate because project d5a66dfea4c5492da1ac3e1f76cbe35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:12:24.961835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.962475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cfa185a0-68ee-4080-9337-e4696a92bec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:24.963999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1189/2415] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:12:24 2022] DELETE /identity/v3/projects/d5a66dfea4c5492da1ac3e1f76cbe35d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:12:24.969009 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:24.969468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:24.969581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:24.990359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:24.991555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:25.041085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:25.049161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1227/2416] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:25.058831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.059113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.059391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.079159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:25.080382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:25.088457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1190/2417] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:25.092787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.102553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b0e37-f13b-46c4-bde2-f8eceaa968d6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.107208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.107803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.108082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.108396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.109037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:12:25.109341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.111036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.123781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1228/2418] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:12:25.130813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.139949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e23b1-1ae6-40d7-af83-bffc0f0f0db0 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.145099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.145782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.146057 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.146327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.146971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.148897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.241836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1191/2419] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:12:25 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:12:25.246002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.257103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae4a43-e605-4fe8-a712-f18f2620200e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.313086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.313711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.313993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.314267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.314931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.317733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.405664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1229/2420] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:12:25 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:12:25.409844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.420840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357759b-dc17-454c-ba02-700b888fbb60 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.425515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.426810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.427115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.427387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.428035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.429733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.439756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1192/2421] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:12:25.443744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.453748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a4d60-2d59-461d-ba36-7e6f8080dbb8 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.458589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.459201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.459507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.459786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.460482 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.462095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496212-b565-4777-81d0-fa69a37c2c02 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.473378 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1230/2422] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:12:25.477656 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.487466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823722a3-45f0-408f-a062-33aa6aa9109f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.493445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.494063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.494339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.494616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] PATH_INFO: `/v3/projects/7d495b3e011c49388cc8cff98aeb1d73/users/b67df0ce7f634db48cbd63d2bb0e5631/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.509065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d495b3e011c49388cc8cff98aeb1d73, user_id=b67df0ce7f634db48cbd63d2bb0e5631, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.511781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.594438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1193/2423] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:12:25 2022] PUT /identity/v3/projects/7d495b3e011c49388cc8cff98aeb1d73/users/b67df0ce7f634db48cbd63d2bb0e5631/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:12:25.599923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.604434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.604434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.621430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:25.622490 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:25.685342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1231/2424] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:25.694302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:25.695822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:25.711280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:25.757666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17381-f567-4d6e-b15e-6f1a546e9ffb admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:25.761212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:25.761940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:25.762100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:25.762369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:25.773417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:25.774880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f27d7-b500-487e-9856-7027ed5f871d service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:25.785174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1194/2425] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:30.857698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:30.883098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f18bf4-4b7b-4de8-afff-a5ba1fe1f377 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:30.890252 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:30.891072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:30.893754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:30.895046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:30.901595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:30.918832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:30.921123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ec68a-1865-4a4f-a6e0-40d6736c1726 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:30.944751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1232/2426] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:31.544488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.545399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.545680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.547083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1195/2427] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:12:31.555013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.556332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.556332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.584573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:31.587873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:31.595368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] MFA Rules not processed for user `0c6b4345e6cd418f8b2a3b6e0b054ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:31.598568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:31.609183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1233/2428] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:12:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:31.623928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:31.627686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:31.641132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9483f-db34-4c95-84f8-506946cadc22 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:31.646357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:31.649659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.649659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.649659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.657997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:31.659481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcefc9-caad-4f1f-b65b-8ce4763df92a service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:31.671223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1234/2429] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:31.723564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b455a-be0b-474b-8076-99c18744d2ec None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:31.737839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:31.741434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.741434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.741434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.743613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:31.774165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:31.775976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:31.789922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd336b-117c-47d6-acf5-28931c0f1cec None None] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.790240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd336b-117c-47d6-acf5-28931c0f1cec None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.790479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd336b-117c-47d6-acf5-28931c0f1cec None None] PATH_INFO: `/` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.791831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1235/2430] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 25 03:12:31.792572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1196/2431] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:31.797995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c1ef4-11c5-42f3-a73c-e1212f9b6812 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.798258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c1ef4-11c5-42f3-a73c-e1212f9b6812 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.798547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c1ef4-11c5-42f3-a73c-e1212f9b6812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.834638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-460c1ef4-11c5-42f3-a73c-e1212f9b6812 None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:31.836627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-460c1ef4-11c5-42f3-a73c-e1212f9b6812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:31.850189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1236/2432] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:12:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:31.861417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:31.874158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684200-c00d-4ea3-b1b4-5ab2162ec8d7 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:31.878510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:31.879487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:31.880714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:31.880714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:31.908235 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:31.909732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:31.932382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1197/2433] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:32.391605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993731-00fb-4ac3-8ec3-538265d88c61 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:32.392193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993731-00fb-4ac3-8ec3-538265d88c61 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:32.392193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993731-00fb-4ac3-8ec3-538265d88c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:32.424294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-82993731-00fb-4ac3-8ec3-538265d88c61 None None] MFA Rules not processed for user `b67df0ce7f634db48cbd63d2bb0e5631`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:32.425616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-82993731-00fb-4ac3-8ec3-538265d88c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:32.439767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:32.497092 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1237/2434] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:12:32.511909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:32.525801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82823a-7f7c-4411-82cf-1ce2fa6b1658 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:32.535035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:32.536706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:32.537178 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:32.537495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:32.547308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:32.551356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:32.559343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1198/2435] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:35.895351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32cf23-1d39-4293-8d0f-e15b220531ce None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:35.896432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32cf23-1d39-4293-8d0f-e15b220531ce None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:35.896904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32cf23-1d39-4293-8d0f-e15b220531ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:35.944423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-0a32cf23-1d39-4293-8d0f-e15b220531ce None None] MFA Rules not processed for user `b67df0ce7f634db48cbd63d2bb0e5631`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:35.957421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0a32cf23-1d39-4293-8d0f-e15b220531ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:35.975188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1238/2436] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:40.173197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:40.175200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.198215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ce00-4e42-43b1-9d68-97f3e9af8632 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:40.206191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:40.206191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:40.206191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:40.206191 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] PATH_INFO: `/v3/users/be1baf177c664022ac3e872df3a3cd27` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:40.209012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1baf177c664022ac3e872df3a3cd27)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:40.211427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:40.479992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1baf177c664022ac3e872df3a3cd27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:40.480793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1baf177c664022ac3e872df3a3cd27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:40.488366 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1baf177c664022ac3e872df3a3cd27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:40.590845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1baf177c664022ac3e872df3a3cd27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:40.591742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1baf177c664022ac3e872df3a3cd27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:40.598233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:40.610427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1199/2437] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:12:40 2022] DELETE /identity/v3/users/be1baf177c664022ac3e872df3a3cd27 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:12:40.879614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:40.880715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:40.881393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:40.908060 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:40.909569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.975290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:40.977041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:40.993273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1239/2438] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:12:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:41.085480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752646b-9756-4b0a-a2cd-9b2dfa493df3 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:41.086055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:41.086786 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.087138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.087478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] PATH_INFO: `/v3/projects/aa73d487b3884350889e14e0b018e0f1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.088701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa73d487b3884350889e14e0b018e0f1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:41.090618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:41.203952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:41.205436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:41.264079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa73d487b3884350889e14e0b018e0f1', 'request_id': 'req-19bcc677-4a77-45cd-be5b-4c52cfd172ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:41.272013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa73d487b3884350889e14e0b018e0f1', 'request_id': 'req-19bcc677-4a77-45cd-be5b-4c52cfd172ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:41.311502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fa71e-b352-4f4a-a6f8-1dd000bc20c9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:41.397383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=100 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:41.400666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa73d487b3884350889e14e0b018e0f1', 'request_id': 'req-19bcc677-4a77-45cd-be5b-4c52cfd172ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:41.420234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:41.421451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.421966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.422460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.427243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] The token cache is being invalidate because project aa73d487b3884350889e14e0b018e0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:12:41.428900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:41.430298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-19bcc677-4a77-45cd-be5b-4c52cfd172ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:12:41.434727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1200/2439] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:12:40 2022] DELETE /identity/v3/projects/aa73d487b3884350889e14e0b018e0f1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:12:41.442044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.442380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.443269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.489918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:41.489918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:41.517033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:41.519739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525e8b5-7ecb-4cf7-8ef1-1849b0d34c3c service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:41.528959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1240/2440] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:12:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:12:41.573921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1201/2441] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:41.584293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.584842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.585460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.610435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:41.613551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:41.623497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1241/2442] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:41.627718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:41.638979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809d7af-20ea-4652-8b24-d9cb4c3a3763 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:41.645644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:41.647115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.647594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.648015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.648952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:12:41.649776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:41.651588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:41.675081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1202/2443] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:12:41.679802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:41.692079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b82669-5e2a-43b4-bc7f-5e325e27ba6f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:41.701330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:41.701954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.702231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.702501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.703140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:41.705208 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:41.842535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1242/2444] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:12:41 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:12:41.848203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:41.856177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:41.863519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c8613-14d0-4454-9234-03ca1fe0c95d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:41.968478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:41.969349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:41.969941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:41.970289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:41.971110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:41.973493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.072601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1203/2445] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:12:41 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:12:42.073548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:42.101940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595e72-c0ad-46e1-9a8f-860fd2b32275 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:42.110272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:42.112390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.113171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.113917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.115389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:42.117334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.130091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1243/2446] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:12:42.137324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:42.152169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dddc7-c935-4cae-8ae4-07b5675aa330 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:42.160695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:42.161451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.161736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.162040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.162785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:42.164768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c7c69-7250-4585-8f68-1296fa4648ea admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.190692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:42.191925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1204/2447] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:12:42.209226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d794-0c58-4992-899a-145b13125f25 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:42.219904 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:42.220721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.221050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.221636 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] PATH_INFO: `/v3/projects/2a2ed10f29ea4a90a7cf7d68756f00b1/users/f2d592851e374bbbb6d63c9bcb2ee942/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.258485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2ed10f29ea4a90a7cf7d68756f00b1, user_id=f2d592851e374bbbb6d63c9bcb2ee942, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:42.265248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.519494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1244/2448] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:12:42 2022] PUT /identity/v3/projects/2a2ed10f29ea4a90a7cf7d68756f00b1/users/f2d592851e374bbbb6d63c9bcb2ee942/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:12:42.537855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd3ed6-8be0-488d-97bd-d8cf29baf879 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.538427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd3ed6-8be0-488d-97bd-d8cf29baf879 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.538763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd3ed6-8be0-488d-97bd-d8cf29baf879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.598704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-26dd3ed6-8be0-488d-97bd-d8cf29baf879 None None] MFA Rules not processed for user `f2d592851e374bbbb6d63c9bcb2ee942`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:12:42.601101 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-26dd3ed6-8be0-488d-97bd-d8cf29baf879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:42.746438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1205/2449] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:12:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:12:42.757621 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:12:42.759638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:12:42.886142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065957d-0392-4c96-a8d6-5ba6e53c387c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:12:42.890484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:12:42.891131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:12:42.891428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:12:42.891725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:12:42.905352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:12:42.910706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c093e44-43cc-48d8-ad92-c10412d7fa7c service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:12:42.922485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1245/2450] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:12:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:12:52.736683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:12:52.917470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=81,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:04.577237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:04.577712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:04.577844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] PATH_INFO: `/` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:04.579346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1206/2451] 10.222.0.75 () {54 vars in 921 bytes} [Thu Aug 25 03:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 25 03:13:04.585510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:04.585994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:04.590016 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:04.607263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] MFA Rules not processed for user `867d21f5f80d4cc48cc53e0b89c7f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:04.608549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:04.691897 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1246/2452] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Aug 25 03:13:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:04.703148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:04.704941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:04.765820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196448a-63d0-4f6f-a48d-d8c8d94d5893 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:04.769420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:04.770024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:04.770306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:04.770580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:04.780661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:04.782111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67b820-7848-4683-ac83-f35a66daa789 service glance] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:04.793914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1207/2453] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:05.388720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:05.461476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa247e7-4bc5-462b-a2c1-5b3004b3e11a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:05.467864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:05.468602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:05.468893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:05.469169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] PATH_INFO: `/v3/users/f2d592851e374bbbb6d63c9bcb2ee942` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:05.470305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d592851e374bbbb6d63c9bcb2ee942)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:05.472572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:05.754391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d592851e374bbbb6d63c9bcb2ee942'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:05.755494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d592851e374bbbb6d63c9bcb2ee942'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:05.763651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d592851e374bbbb6d63c9bcb2ee942'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:05.869572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d592851e374bbbb6d63c9bcb2ee942'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:05.870128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d592851e374bbbb6d63c9bcb2ee942'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:05.888401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1247/2454] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:05 2022] DELETE /identity/v3/users/f2d592851e374bbbb6d63c9bcb2ee942 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:05.893439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:05.893917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:05.894416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:05.916108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:05.917949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:05.993326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1208/2455] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:06.003587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:06.005067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:06.019002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:06.063885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593ced-5478-4783-9b1a-8004a91be8fd admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:06.067810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:06.068523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:06.068805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:06.069080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:06.082664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:06.085942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863633b-811a-4207-b3d8-d3c775ae4a16 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:06.101675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1248/2456] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:06.552538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:06.553973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:06.566185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ee55-8773-4a2a-8104-74fc7f6ab63f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:06.571038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:06.575614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:06.575614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:06.575614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] PATH_INFO: `/v3/projects/2a2ed10f29ea4a90a7cf7d68756f00b1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:06.575614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2ed10f29ea4a90a7cf7d68756f00b1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:06.575962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:06.710644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2ed10f29ea4a90a7cf7d68756f00b1', 'request_id': 'req-7393ce63-7518-45cc-af69-7ee14c2601ce'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:06.715874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2ed10f29ea4a90a7cf7d68756f00b1', 'request_id': 'req-7393ce63-7518-45cc-af69-7ee14c2601ce'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:06.823912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2ed10f29ea4a90a7cf7d68756f00b1', 'request_id': 'req-7393ce63-7518-45cc-af69-7ee14c2601ce'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:06.837428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] The token cache is being invalidate because project 2a2ed10f29ea4a90a7cf7d68756f00b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:06.837590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:06.838244 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:06.839742 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1209/2457] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:06 2022] DELETE /identity/v3/projects/2a2ed10f29ea4a90a7cf7d68756f00b1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:07.961578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc2213-180c-44b7-bb99-75cd581976a9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:07.962049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc2213-180c-44b7-bb99-75cd581976a9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:07.962145 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc2213-180c-44b7-bb99-75cd581976a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:07.981664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f0cc2213-180c-44b7-bb99-75cd581976a9 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:07.982861 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f0cc2213-180c-44b7-bb99-75cd581976a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:08.040272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1249/2458] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:13:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:08.052845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:08.054266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:08.085989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:08.127385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393ce63-7518-45cc-af69-7ee14c2601ce admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:08.132963 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:08.133603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:08.133883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:08.134182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:08.142849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:08.144525 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:08.152966 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1210/2459] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:13:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:13:18.034817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=46 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:18.279713 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=20 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:27.231784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:27.232295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:27.232509 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:27.254722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:27.256238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:27.264024 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1250/2460] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:27.277110 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:27.288907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb431c0a-75ea-4749-8f22-6845ef3aeb0d service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:27.294008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:27.294605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:27.294886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:27.295159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:27.303820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:27.305439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:27.313537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1211/2461] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:13:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:13:29.011032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:29.012450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:29.069305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36600940-3373-4ab3-aa08-b763f3e45217 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:29.073042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:29.073638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:29.073915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:29.074186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:29.075863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:29.138510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:29.139925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:29.149875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1251/2462] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:33.047000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:33.048807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:33.063436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487c61b-5a12-42e9-9c00-0b4eb448f862 service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:33.068340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:33.068965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:33.069243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:33.069514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] PATH_INFO: `/v3/users/804df9333a474723968f97eb9d6bfc69` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:33.070637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=804df9333a474723968f97eb9d6bfc69)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:33.072706 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:33.366737 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804df9333a474723968f97eb9d6bfc69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:33.367391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804df9333a474723968f97eb9d6bfc69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:33.374651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804df9333a474723968f97eb9d6bfc69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:33.553414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804df9333a474723968f97eb9d6bfc69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:33.554040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-e4698a6d-24da-448c-949e-1899a39f83ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804df9333a474723968f97eb9d6bfc69'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:33.572314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1212/2463] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:33 2022] DELETE /identity/v3/users/804df9333a474723968f97eb9d6bfc69 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:33.853304 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:33.864576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57351493-f346-4cbb-b4eb-e75b7670d4cd service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:33.930663 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:33.931287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:33.931572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:33.931847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] PATH_INFO: `/v3/projects/f626c34f12fb4bdab6bd42b05b79742a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:33.933312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f626c34f12fb4bdab6bd42b05b79742a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:33.934901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.100241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:34.103017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f626c34f12fb4bdab6bd42b05b79742a', 'request_id': 'req-d097fd17-d7ab-4fd7-9077-546b1bd99b68'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:34.108586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f626c34f12fb4bdab6bd42b05b79742a', 'request_id': 'req-d097fd17-d7ab-4fd7-9077-546b1bd99b68'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:34.115255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f626c34f12fb4bdab6bd42b05b79742a', 'request_id': 'req-d097fd17-d7ab-4fd7-9077-546b1bd99b68'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:34.301734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] The token cache is being invalidate because project f626c34f12fb4bdab6bd42b05b79742a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:34.302030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:34.302820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-d097fd17-d7ab-4fd7-9077-546b1bd99b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:34.304569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1252/2464] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:33 2022] DELETE /identity/v3/projects/f626c34f12fb4bdab6bd42b05b79742a => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:34.309384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.309846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.309957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.330515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:34.331738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:34.390429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1213/2465] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:34.400431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.400719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.400992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.426224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:34.426224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:34.432781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1253/2466] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:34.437054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.447079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bd993-f62a-4319-8972-8e0db3ca7cd7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.451774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.452407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.452693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.452969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.453614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:13:34.453924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.455688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.467137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1214/2467] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:13:34.471885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.481476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba6b74-407f-43b0-b382-dae950fa2d8b None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.487008 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.487629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.487906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.488188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.488833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.490728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.645635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1254/2468] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:34 2022] POST /identity/v3/projects => generated 363 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:34.646823 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.657847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fd99-5d2a-4cea-8038-1f553c6f2ea9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.701480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:34.714674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.715284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.715565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.715839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.716507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.718431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.865014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1215/2469] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:34 2022] POST /identity/v3/users => generated 282 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:13:34.869229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.880725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c045-dbe5-4719-96fb-556af6fe720c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.885262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.885853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.886131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.886399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.887031 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.888640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.898773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1255/2470] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:34.909439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.919928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526a49-089d-42eb-84fb-92e06c3b78f3 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.925636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.926254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.926549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.926820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.927474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.929209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04012c-d6b1-49eb-b487-2209a9ae6d61 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:34.941234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1216/2471] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:34.945495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:34.955971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19e950-5f08-4e97-9c26-857fee242fbf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:34.960667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:34.961321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:34.961601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:34.961875 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] PATH_INFO: `/v3/projects/85ee494130d046b3bc37f9d43b9ec4d7/users/be70282bc7bb43d5bf5402aeaf804f73/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:34.976835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ee494130d046b3bc37f9d43b9ec4d7, user_id=be70282bc7bb43d5bf5402aeaf804f73, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:34.980787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.074485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1256/2472] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:34 2022] PUT /identity/v3/projects/85ee494130d046b3bc37f9d43b9ec4d7/users/be70282bc7bb43d5bf5402aeaf804f73/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:35.086118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.086411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.086686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.116275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] MFA Rules not processed for user `be70282bc7bb43d5bf5402aeaf804f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:35.117734 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:35.181249 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1217/2473] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:35.186000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.249205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253ba2-186a-49ef-ae5f-86aea3d3b64a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.254414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.255025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.255367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.255787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.256556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.258683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.426399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1257/2474] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:35 2022] POST /identity/v3/projects => generated 361 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:13:35.431311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.442715 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88becd63-df67-41ca-81a9-e47d539a4ce5 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.499695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.500248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.500531 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.500809 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.501429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.503660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.637615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1218/2475] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:35 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:13:35.639567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.650249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084af0d-2ea7-45e9-b93d-caa18022de43 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.656978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.656978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.656978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.656978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.657321 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.658979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.670165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1258/2476] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:35.675225 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.684949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e061541-7938-4590-a63f-c61184ca1d1a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.691046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.691719 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.692041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.693498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] PATH_INFO: `/v3/projects/d83a233748a8423d89f3165f3ae53ebf/users/7ec8d76211034429aa5e1758635dcf3c/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.710274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83a233748a8423d89f3165f3ae53ebf, user_id=7ec8d76211034429aa5e1758635dcf3c, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.713008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.794568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1219/2477] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:35 2022] PUT /identity/v3/projects/d83a233748a8423d89f3165f3ae53ebf/users/7ec8d76211034429aa5e1758635dcf3c/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:35.798468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.809246 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab36bab-00d3-4c3c-9dab-015b36fada10 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.853764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:35.868845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.869459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.870017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.870017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.870648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.872229 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba9cc-ee4a-4928-bd3f-8da0824ceccb admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.882521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1259/2478] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:35.886733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:35.897138 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1f01-c251-432e-af35-c1776be1c069 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:35.901971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:35.902629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.902911 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.903587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] PATH_INFO: `/v3/projects/d83a233748a8423d89f3165f3ae53ebf/users/7ec8d76211034429aa5e1758635dcf3c/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.905162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83a233748a8423d89f3165f3ae53ebf, user_id=7ec8d76211034429aa5e1758635dcf3c, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:35.907840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:35.919088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1220/2479] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:35 2022] PUT /identity/v3/projects/d83a233748a8423d89f3165f3ae53ebf/users/7ec8d76211034429aa5e1758635dcf3c/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:35.930134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:35.930515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:35.930868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:35.959855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] MFA Rules not processed for user `7ec8d76211034429aa5e1758635dcf3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:35.961192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:36.029106 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1260/2480] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:36.041209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:36.042928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:36.053717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:36.101457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f5a4-9a05-49a6-a6e2-ed79e7082494 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:36.105183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:36.105777 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:36.106056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:36.106453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:36.116003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:36.117461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc334f-5fe4-4c8e-bc88-61a64da9abfe service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:36.127371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1221/2481] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:36.579389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:36.581540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:36.593192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b39478-f781-4781-ac16-e298c3f320b5 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:36.597302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:36.597891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:36.598166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:36.598436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:36.655182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:36.656655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:36.666015 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1261/2482] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:46.999841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=23 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:47.062225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=71 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:49.396019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89417edd-de63-4c02-b324-a146a5ddc09f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:49.396651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89417edd-de63-4c02-b324-a146a5ddc09f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:49.396651 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89417edd-de63-4c02-b324-a146a5ddc09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:49.415810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-89417edd-de63-4c02-b324-a146a5ddc09f None None] MFA Rules not processed for user `b67df0ce7f634db48cbd63d2bb0e5631`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:49.417127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-89417edd-de63-4c02-b324-a146a5ddc09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:49.473182 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1222/2483] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:49.524615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:49.600938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de618f1-0991-42b0-add2-3f5560d9a783 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:49.605616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:49.606254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:49.606535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:49.606819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] PATH_INFO: `/v3/users/be70282bc7bb43d5bf5402aeaf804f73` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:49.607951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] RBAC: Authorizing `identity:delete_user(user_id=be70282bc7bb43d5bf5402aeaf804f73)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:49.609925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:49.856209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be70282bc7bb43d5bf5402aeaf804f73'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:49.856787 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be70282bc7bb43d5bf5402aeaf804f73'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:49.865077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be70282bc7bb43d5bf5402aeaf804f73'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:49.948903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be70282bc7bb43d5bf5402aeaf804f73'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:49.949602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be70282bc7bb43d5bf5402aeaf804f73'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:49.968180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1262/2484] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:49 2022] DELETE /identity/v3/users/be70282bc7bb43d5bf5402aeaf804f73 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:49.972695 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:49.973048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:49.973395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:49.993014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:49.994227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.056581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1223/2485] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:50.067249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:50.068873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.126449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014bc2-4033-48b0-93d2-a4950aad37af admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:50.130264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:50.130868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:50.131149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:50.131423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:50.140346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:50.141843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:50.151572 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1263/2486] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:50.688413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:50.689218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:50.708673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd613-0643-4ae2-9450-867df89b1d9c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:50.723593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:50.724265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:50.724547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:50.724818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] PATH_INFO: `/v3/projects/85ee494130d046b3bc37f9d43b9ec4d7` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:50.725953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ee494130d046b3bc37f9d43b9ec4d7)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:50.727918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:50.908063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ee494130d046b3bc37f9d43b9ec4d7', 'request_id': 'req-822d6021-58e1-4de4-9e72-f2728f47da89'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:50.914260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ee494130d046b3bc37f9d43b9ec4d7', 'request_id': 'req-822d6021-58e1-4de4-9e72-f2728f47da89'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.005097 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ee494130d046b3bc37f9d43b9ec4d7', 'request_id': 'req-822d6021-58e1-4de4-9e72-f2728f47da89'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.016900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] The token cache is being invalidate because project 85ee494130d046b3bc37f9d43b9ec4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:51.017518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.017860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.019804 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1224/2487] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:50 2022] DELETE /identity/v3/projects/85ee494130d046b3bc37f9d43b9ec4d7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:51.023886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:51.034170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd92ce1-5203-4de3-a54a-aa8cc963dfea service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:51.053565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:51.091475 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:51.092107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:51.092415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:51.092690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] PATH_INFO: `/v3/users/7ec8d76211034429aa5e1758635dcf3c` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:51.093784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec8d76211034429aa5e1758635dcf3c)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:51.095704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:51.290421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec8d76211034429aa5e1758635dcf3c'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.291261 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec8d76211034429aa5e1758635dcf3c'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.298299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec8d76211034429aa5e1758635dcf3c'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.473626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec8d76211034429aa5e1758635dcf3c'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.474259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9c6e33bd-62bd-4f5e-84a3-c8b0e84b4d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec8d76211034429aa5e1758635dcf3c'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:51.492614 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1264/2488] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:51 2022] DELETE /identity/v3/users/7ec8d76211034429aa5e1758635dcf3c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:52.010141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.010141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:52.023536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d6021-58e1-4de4-9e72-f2728f47da89 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.056430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:52.093440 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.094000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.094287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.094714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] PATH_INFO: `/v3/projects/d83a233748a8423d89f3165f3ae53ebf` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.096000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83a233748a8423d89f3165f3ae53ebf)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.098173 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.226728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83a233748a8423d89f3165f3ae53ebf', 'request_id': 'req-d03ccef9-ea26-42f3-836d-bbd68e6348fd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:52.232618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83a233748a8423d89f3165f3ae53ebf', 'request_id': 'req-d03ccef9-ea26-42f3-836d-bbd68e6348fd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:52.359321 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83a233748a8423d89f3165f3ae53ebf', 'request_id': 'req-d03ccef9-ea26-42f3-836d-bbd68e6348fd'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:52.371850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] The token cache is being invalidate because project d83a233748a8423d89f3165f3ae53ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:52.372197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:52.373268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-d03ccef9-ea26-42f3-836d-bbd68e6348fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:52.374843 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1225/2489] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:52 2022] DELETE /identity/v3/projects/d83a233748a8423d89f3165f3ae53ebf => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:52.379852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.380286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.380538 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.400838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:52.402169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:52.466186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1265/2490] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:52.478613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.478901 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.479179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.499818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:52.501061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:52.509569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1226/2491] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:52.513882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.523551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6b011-3072-45be-bf9c-d9c3d010e45d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.529100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.529100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.529100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.529435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.529942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:13:52.530247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.531846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.543846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1266/2492] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:13:52.548526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.558733 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7e7f7-aec0-4f40-997b-8691a447dc88 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.569351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.569351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.569351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.569351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.569784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.569784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.665567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1227/2493] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:52 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:52.669337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.680255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b6d7-4b38-4331-a166-17b69cde3cd1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.707747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:52.739392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.740000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.740305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.740615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.741298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.743218 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.892071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1267/2494] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:52 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:52.896837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.907544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472272-e6d7-4dac-aaa9-6b7f36f9a856 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.913358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.913870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.914162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.914424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.915071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.916769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.927603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1228/2495] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:52.931584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.941954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b9d8-10db-4846-809f-39280e311eda admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.946587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.947190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.947467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.947738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:52.948412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:52.949993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fcf29-74c5-44f7-b068-83ba4ffac27c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:52.961197 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1268/2496] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:13:52.965438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:52.976874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec85e2-0644-4ee9-a724-446cd3abd439 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:52.981428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:52.982081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:52.982359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:52.982631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] PATH_INFO: `/v3/projects/9a0a3ec3cfbd4f5bbfa04e2dbfde3165/users/e288aa3eb96a4cd497d5efa414002a27/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.000132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0a3ec3cfbd4f5bbfa04e2dbfde3165, user_id=e288aa3eb96a4cd497d5efa414002a27, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:53.003376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:53.133347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1229/2497] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:52 2022] PUT /identity/v3/projects/9a0a3ec3cfbd4f5bbfa04e2dbfde3165/users/e288aa3eb96a4cd497d5efa414002a27/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:53.144209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:53.144546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:53.144886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.173835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] MFA Rules not processed for user `e288aa3eb96a4cd497d5efa414002a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:53.175291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:53.241430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1269/2498] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:53.245761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:53.426164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:53.433083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf286-3697-484c-8e0d-c27ad0294e72 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:53.437740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:53.438389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:53.438663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:53.438933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.440165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:53.442236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:53.703731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1230/2499] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:53 2022] POST /identity/v3/projects => generated 385 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:13:53.707816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:53.724071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb9939-1c50-44d8-a070-278cca32bced None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:53.789845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:53.790554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:53.790909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:53.791267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.791900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:53.793840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:53.974980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1270/2500] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:53 2022] POST /identity/v3/users => generated 299 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:53.975987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:53.986293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1103e-3e98-4575-b1a9-1916b6dc94bd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:53.991855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:53.992569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:53.992876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:53.993143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:53.993794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:53.995405 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.007410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1231/2501] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:54.011789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:54.023775 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56215-0147-47ae-a085-ff2e0d249c4b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:54.028558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:54.029235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.029576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.029784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] PATH_INFO: `/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64/users/55e791d55341421eac2a8ba578e57fd7/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.047796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9b6b41ae7d448eaefebcf16e27bd64, user_id=55e791d55341421eac2a8ba578e57fd7, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:54.051238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.154661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1271/2502] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:54 2022] PUT /identity/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64/users/55e791d55341421eac2a8ba578e57fd7/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:13:54.159150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:54.169063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0898164-5b8a-4c0e-9230-7969042f95a9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:54.225011 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:54.225537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.225933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.226163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.226831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:54.228606 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed168c-0212-4a84-b20c-a0efb70a9878 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.239152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1232/2503] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:54.242857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:54.248674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:54.254189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07f97-219a-4f5c-92ab-15060af82d1c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:54.259032 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:54.259686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.260057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.260368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] PATH_INFO: `/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64/users/55e791d55341421eac2a8ba578e57fd7/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.262392 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9b6b41ae7d448eaefebcf16e27bd64, user_id=55e791d55341421eac2a8ba578e57fd7, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:54.265088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.277181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1272/2504] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:54 2022] PUT /identity/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64/users/55e791d55341421eac2a8ba578e57fd7/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:54.287975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.288298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.288565 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.315243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] MFA Rules not processed for user `55e791d55341421eac2a8ba578e57fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:54.316617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:54.393707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1233/2505] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:13:54.405791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:54.407584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:54.467134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d266-67f6-49d2-84f2-3a5532e2a1f3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:54.471568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:54.472524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.472807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.473085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.482701 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:54.484094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e1353-ea41-4313-8430-f76600a72233 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.500864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1273/2506] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:54.569964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:54.632834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:54.644698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8eb12-d2fe-4275-8041-d11b0a2d779a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:54.649374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:54.650367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:54.650367 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:54.650607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] PATH_INFO: `/v3/users/e288aa3eb96a4cd497d5efa414002a27` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:54.651721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e288aa3eb96a4cd497d5efa414002a27)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:54.653708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:54.942527 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e288aa3eb96a4cd497d5efa414002a27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:54.943362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e288aa3eb96a4cd497d5efa414002a27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:54.950438 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e288aa3eb96a4cd497d5efa414002a27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.022419 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e288aa3eb96a4cd497d5efa414002a27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.023504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e288aa3eb96a4cd497d5efa414002a27'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.047160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1234/2507] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:54 2022] DELETE /identity/v3/users/e288aa3eb96a4cd497d5efa414002a27 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:55.052715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:55.053209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:55.053325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:55.073206 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:55.074411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:55.136297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1274/2508] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:55.146393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:55.148477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:55.206884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f884-a5ab-4f0e-b257-9323a928acaa admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:55.210627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:55.211234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:55.211511 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:55.211779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:55.221560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:55.223287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:55.233086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1235/2509] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:55.755508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:55.760540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:55.769396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b69053-c283-4dfc-a1b8-d9bae259e96c None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:55.774004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:55.774708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:55.774993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:55.775297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] PATH_INFO: `/v3/projects/9a0a3ec3cfbd4f5bbfa04e2dbfde3165` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:55.776559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0a3ec3cfbd4f5bbfa04e2dbfde3165)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:55.778501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:55.893507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0a3ec3cfbd4f5bbfa04e2dbfde3165', 'request_id': 'req-fe4b1b33-5b94-48aa-a11a-b1b5af523232'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.899100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0a3ec3cfbd4f5bbfa04e2dbfde3165', 'request_id': 'req-fe4b1b33-5b94-48aa-a11a-b1b5af523232'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.906773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0a3ec3cfbd4f5bbfa04e2dbfde3165', 'request_id': 'req-fe4b1b33-5b94-48aa-a11a-b1b5af523232'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:55.993468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] The token cache is being invalidate because project 9a0a3ec3cfbd4f5bbfa04e2dbfde3165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:56.003502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.003502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.003502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1275/2510] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:55 2022] DELETE /identity/v3/projects/9a0a3ec3cfbd4f5bbfa04e2dbfde3165 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:56.006210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:56.015773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fa4b4-6f06-4b7d-9ed6-c965c38176c3 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:56.073668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:56.074222 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:56.074506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:56.074783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] PATH_INFO: `/v3/users/55e791d55341421eac2a8ba578e57fd7` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:56.075895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e791d55341421eac2a8ba578e57fd7)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:56.077914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:56.091897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:56.355304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e791d55341421eac2a8ba578e57fd7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.355964 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e791d55341421eac2a8ba578e57fd7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.362624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e791d55341421eac2a8ba578e57fd7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.605087 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e791d55341421eac2a8ba578e57fd7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.605763 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-2e00c4c1-763f-43a7-8ce5-d3e2c896e2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e791d55341421eac2a8ba578e57fd7'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:56.624445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1236/2511] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:13:56 2022] DELETE /identity/v3/users/55e791d55341421eac2a8ba578e57fd7 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:57.104531 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:57.106004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:57.129837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1b33-5b94-48aa-a11a-b1b5af523232 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:57.142483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:57.193769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:57.194433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.194721 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.194999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] PATH_INFO: `/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.196126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9b6b41ae7d448eaefebcf16e27bd64)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:57.198072 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:57.370969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9b6b41ae7d448eaefebcf16e27bd64', 'request_id': 'req-16ead472-e47c-4cb8-881d-1eeb46a01bf1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:57.377109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9b6b41ae7d448eaefebcf16e27bd64', 'request_id': 'req-16ead472-e47c-4cb8-881d-1eeb46a01bf1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:57.384935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9b6b41ae7d448eaefebcf16e27bd64', 'request_id': 'req-16ead472-e47c-4cb8-881d-1eeb46a01bf1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:57.530386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] The token cache is being invalidate because project 3e9b6b41ae7d448eaefebcf16e27bd64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:13:57.530677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:57.531299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-16ead472-e47c-4cb8-881d-1eeb46a01bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:13:57.532813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1276/2512] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:13:57 2022] DELETE /identity/v3/projects/3e9b6b41ae7d448eaefebcf16e27bd64 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:57.538410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.546472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.546472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.565575 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:57.566818 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:57.625069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1237/2513] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:57.634479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.634765 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.635087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.654922 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:57.656120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:57.664101 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1277/2514] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:57.668426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:57.678029 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ca26a-d984-4945-9063-56d5e5c96aa1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:57.682611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:57.683201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.683494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.683795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.684470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:13:57.684774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:57.686371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:57.698905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1238/2515] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:13:57.703247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:57.712983 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d05c0a-92bb-4640-b541-92ce7b9f518d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:57.717782 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:57.718382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.718681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.718953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.719584 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:57.722266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:57.879191 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1278/2516] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:57 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:13:57.883784 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:57.895293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202cf21-40fd-4fb7-9c05-4fbd94c33c62 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:57.969834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:57.970432 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:57.970772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:57.971660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:57.972318 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:57.974218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.125205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1239/2517] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:57 2022] POST /identity/v3/users => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:58.129521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.140049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca045f-26b8-492e-85c7-3fdc305d0dd4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.145647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.146379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.146626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.146902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.147548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.149263 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.160134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1279/2518] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:58.164036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.174429 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a2af-bd33-40f9-a9d4-a3e9b954acf7 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.179019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.179640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.179921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.180219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.180857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.182458 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6b8a5-b389-42f3-b383-8bc46fd52406 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.184644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:58.200789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1240/2519] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:58.203284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.214241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebba07-c89e-4578-92ae-6926c9aabdf9 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.218883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.219554 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.219835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.220110 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] PATH_INFO: `/v3/projects/f6ef5e5459924395864e25aae8d31f80/users/5e2c16b3cfb04a08863ccf80518a9a1f/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.236041 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ef5e5459924395864e25aae8d31f80, user_id=5e2c16b3cfb04a08863ccf80518a9a1f, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.240574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.348635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1280/2520] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:58 2022] PUT /identity/v3/projects/f6ef5e5459924395864e25aae8d31f80/users/5e2c16b3cfb04a08863ccf80518a9a1f/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:58.359112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.359397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.359668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.387260 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] MFA Rules not processed for user `5e2c16b3cfb04a08863ccf80518a9a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:58.388512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:58.451111 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1241/2521] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:13:58.456205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.499245 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:58.516118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c87b3c-a254-4655-94af-97d1e765e689 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.521607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.522211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.522492 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.522794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.523409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.525331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.623877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1281/2522] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:13:58 2022] POST /identity/v3/projects => generated 397 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:58.628546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.639502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b78b-a418-4f69-b6f1-ad087336dcd4 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.695398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.695996 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.696306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.696577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.697190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.699094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.839410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1242/2523] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:13:58 2022] POST /identity/v3/users => generated 305 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:13:58.843350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.853619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228144-e1ae-4888-b2b4-3a1c9799160a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.858125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.870893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.870893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.870893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.870893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.870893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:58.872138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1282/2524] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:58.876647 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:58.887189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f700e-8157-4936-9b8a-ee30fd79b0ea admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:58.891799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:58.892508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:58.892799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:58.893074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] PATH_INFO: `/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46/users/d4b6d92b2d4f4a259d990d6472de9b43/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:58.907302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52f64432a8c4a69bc0fb6d8ec515c46, user_id=d4b6d92b2d4f4a259d990d6472de9b43, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:58.909998 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.011131 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1243/2525] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:58 2022] PUT /identity/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46/users/d4b6d92b2d4f4a259d990d6472de9b43/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:59.015239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:59.025633 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3747-c030-4137-889e-7026828cc3f5 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:59.082299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:59.082910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.083213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.083489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.084132 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:59.085756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6c929-db38-4c19-a2d7-6365c4c4fae2 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.096748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1283/2526] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:13:59.101560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:59.112304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2a2c-6011-47b0-97cc-43b955292166 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:59.116936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:59.117572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.117869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.118118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] PATH_INFO: `/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46/users/d4b6d92b2d4f4a259d990d6472de9b43/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.120046 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52f64432a8c4a69bc0fb6d8ec515c46, user_id=d4b6d92b2d4f4a259d990d6472de9b43, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:59.123068 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.133186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1244/2527] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:13:59 2022] PUT /identity/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46/users/d4b6d92b2d4f4a259d990d6472de9b43/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:13:59.144779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.145287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.145397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.172597 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] MFA Rules not processed for user `d4b6d92b2d4f4a259d990d6472de9b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:13:59.174678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:59.242819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1284/2528] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:13:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:13:59.254297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:59.256039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:13:59.298464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:59.312061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8440-abf3-4128-9e25-6d9d6a4e595e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:59.315840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:59.321473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.321473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.321473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.325905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:59.327308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.336510 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1245/2529] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:13:59.619427 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:13:59.629802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f489c8-4d16-4676-a1a8-8c44cee4ac0a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:13:59.633341 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:13:59.633939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:13:59.634287 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:13:59.634567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:13:59.669305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:13:59.693266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:13:59.694725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd2655-a9df-4f75-972a-be3962570172 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:13:59.703806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1285/2530] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:13:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:05.804002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:05.876903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05638a2b-ae3a-470e-ac31-544193e84cc5 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:05.883386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:05.884663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:05.885054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:05.885435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] PATH_INFO: `/v3/users/5e2c16b3cfb04a08863ccf80518a9a1f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:05.887430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2c16b3cfb04a08863ccf80518a9a1f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:05.889903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:06.159184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2c16b3cfb04a08863ccf80518a9a1f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:06.160243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2c16b3cfb04a08863ccf80518a9a1f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:06.169665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2c16b3cfb04a08863ccf80518a9a1f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:06.308621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2c16b3cfb04a08863ccf80518a9a1f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:06.309329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2c16b3cfb04a08863ccf80518a9a1f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:06.329609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1246/2531] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:05 2022] DELETE /identity/v3/users/5e2c16b3cfb04a08863ccf80518a9a1f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:06.334466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:06.334996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:06.335123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:06.407707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:06.410149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:06.525705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1286/2532] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:06.535574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:06.537130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:06.629081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:06.637961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cfa97-16b3-45b7-8e2f-fc03d6991a79 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:06.642204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:06.642351 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:06.642628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:06.643470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:06.653501 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:06.655821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:06.668327 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1247/2533] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:07.311677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:07.313553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:07.331414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff1717-70c3-4909-a503-76a3c6b27268 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:07.336667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:07.341416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:07.341416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:07.341416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] PATH_INFO: `/v3/projects/f6ef5e5459924395864e25aae8d31f80` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:07.341416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ef5e5459924395864e25aae8d31f80)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:07.342258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:07.627310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ef5e5459924395864e25aae8d31f80', 'request_id': 'req-0b179fff-e206-4a54-90fd-1803b2e07e75'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:07.634232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ef5e5459924395864e25aae8d31f80', 'request_id': 'req-0b179fff-e206-4a54-90fd-1803b2e07e75'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:07.641630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ef5e5459924395864e25aae8d31f80', 'request_id': 'req-0b179fff-e206-4a54-90fd-1803b2e07e75'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:07.643564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:07.787928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] The token cache is being invalidate because project f6ef5e5459924395864e25aae8d31f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:07.788231 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:07.788944 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:07.790489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1287/2534] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:07 2022] DELETE /identity/v3/projects/f6ef5e5459924395864e25aae8d31f80 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:07.794794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:07.804878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a7cd8-5950-4842-ae42-2358d8c2e778 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:07.861496 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:07.862323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:07.862618 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:07.862870 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] PATH_INFO: `/v3/users/d4b6d92b2d4f4a259d990d6472de9b43` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:07.863980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b6d92b2d4f4a259d990d6472de9b43)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:07.865959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:08.184528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b6d92b2d4f4a259d990d6472de9b43'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:08.185256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b6d92b2d4f4a259d990d6472de9b43'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:08.195298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b6d92b2d4f4a259d990d6472de9b43'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:08.370590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b6d92b2d4f4a259d990d6472de9b43'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:08.371247 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8fee1754-cf70-43db-8a64-e509f64dbcdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b6d92b2d4f4a259d990d6472de9b43'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:08.399800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1248/2535] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:07 2022] DELETE /identity/v3/users/d4b6d92b2d4f4a259d990d6472de9b43 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:08.977827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:08.980103 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:08.993506 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179fff-e206-4a54-90fd-1803b2e07e75 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:09.051405 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:09.052035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.052345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.052618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] PATH_INFO: `/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.053722 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52f64432a8c4a69bc0fb6d8ec515c46)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:09.055607 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:09.279375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52f64432a8c4a69bc0fb6d8ec515c46', 'request_id': 'req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:09.284797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52f64432a8c4a69bc0fb6d8ec515c46', 'request_id': 'req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:09.290894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52f64432a8c4a69bc0fb6d8ec515c46', 'request_id': 'req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:09.445932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] The token cache is being invalidate because project a52f64432a8c4a69bc0fb6d8ec515c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:09.446235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:09.446917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cb8687ae-cb54-4e6c-96e5-d5a04b941f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:09.448442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1288/2536] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:08 2022] DELETE /identity/v3/projects/a52f64432a8c4a69bc0fb6d8ec515c46 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:09.453248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.453707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.453820 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.503230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:09.504390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:09.539063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:09.567426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1249/2537] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:09.583028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.583315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.583588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.603835 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:09.605264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:09.613830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1289/2538] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:09.618044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:09.627981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f019263-72de-4fa9-a013-8a587dcefc17 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:09.632513 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:09.633099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.633377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.633640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.634290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:09.634591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:09.636188 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:09.647427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1250/2539] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:09.651949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:09.662259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e258d7-d061-4369-a83c-ada25f238e8d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:09.668209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:09.668209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.668209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.668209 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.668865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:09.670734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:09.795126 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1290/2540] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:09 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:09.800028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:09.809868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac5618-3f64-48ec-9976-979489547dbc admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:09.865362 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:09.865972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:09.866252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:09.866552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:09.867186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:09.869175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:09.998585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1251/2541] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:09 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:10.003056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:10.013423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc73ed-f896-4847-9d65-66e80fedfeba admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:10.018736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:10.019333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.019617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.019924 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.020594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:10.022186 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:10.032006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1291/2542] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:10.036710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:10.047354 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57aee7-275e-4896-8bde-9ff0d843e650 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:10.052104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:10.052745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.053020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.057914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.057914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:10.057914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510ab27-24a6-42df-92bd-28cbe3847e90 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:10.066207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1252/2543] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:10.070466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:10.081088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebd0dd-3bff-40f3-b0d1-feae7d00b109 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:10.086144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:10.086803 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.087082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.087356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] PATH_INFO: `/v3/projects/8577b0a2c2c14534a101878d9c9d4f23/users/de3e52696af343c1a20d940119b5f6b9/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.103003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8577b0a2c2c14534a101878d9c9d4f23, user_id=de3e52696af343c1a20d940119b5f6b9, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:10.105665 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:10.239465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1292/2544] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:10 2022] PUT /identity/v3/projects/8577b0a2c2c14534a101878d9c9d4f23/users/de3e52696af343c1a20d940119b5f6b9/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:10.249979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.250267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.250703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.279335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] MFA Rules not processed for user `de3e52696af343c1a20d940119b5f6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:10.280701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:10.344717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1253/2545] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:10.357615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:10.359338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:10.381087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:10.417578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4bb8-ce6a-4a6b-94c2-0b3a2e87794d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:10.421240 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:10.421847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.422128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.422403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.431234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:10.432693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7e993-5eac-4adc-bb01-7577a28d77f7 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:10.441926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1293/2546] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:10.461936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:10.523309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29cdfa-0221-45f6-bf48-7c2ca052aa62 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:10.528013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:10.528669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.528978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.529250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] PATH_INFO: `/v3/users/de3e52696af343c1a20d940119b5f6b9` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.530407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3e52696af343c1a20d940119b5f6b9)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:10.532399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:10.756290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3e52696af343c1a20d940119b5f6b9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:10.756937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3e52696af343c1a20d940119b5f6b9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:10.764239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3e52696af343c1a20d940119b5f6b9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:10.936161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3e52696af343c1a20d940119b5f6b9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:10.936797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3e52696af343c1a20d940119b5f6b9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:10.955889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1254/2547] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:10 2022] DELETE /identity/v3/users/de3e52696af343c1a20d940119b5f6b9 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:10.960419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:10.960712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:10.960986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:10.980493 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:10.981710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:11.047706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1294/2548] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:11.058357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:11.059760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:11.120281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1f384-016d-44a3-a710-e8eeb6856c00 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:11.124251 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:11.124844 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:11.125120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:11.125388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:11.134959 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:11.137226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3ed93-f19e-4f7e-bfa9-794a837990fe service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:11.147538 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1255/2549] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:11.199371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:11.584505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:11.585913 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:11.598830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ee792-94c6-4fa3-9ae2-74d4c197c5d2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:11.604472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:11.604826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:11.605118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:11.605385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] PATH_INFO: `/v3/projects/8577b0a2c2c14534a101878d9c9d4f23` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:11.606507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8577b0a2c2c14534a101878d9c9d4f23)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:11.608450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:11.737224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8577b0a2c2c14534a101878d9c9d4f23', 'request_id': 'req-f91447ab-ec55-4865-91a1-e80af2141d92'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:11.742844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8577b0a2c2c14534a101878d9c9d4f23', 'request_id': 'req-f91447ab-ec55-4865-91a1-e80af2141d92'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:11.749359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8577b0a2c2c14534a101878d9c9d4f23', 'request_id': 'req-f91447ab-ec55-4865-91a1-e80af2141d92'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:11.833435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] The token cache is being invalidate because project 8577b0a2c2c14534a101878d9c9d4f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:11.833625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:11.834250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f91447ab-ec55-4865-91a1-e80af2141d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:11.835826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1295/2550] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:11 2022] DELETE /identity/v3/projects/8577b0a2c2c14534a101878d9c9d4f23 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:11.840739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:11.841236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:11.841333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:11.861899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:11.863593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:11.912758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:11.923145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1256/2551] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:11.937346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:11.937627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:11.937950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:11.959463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:11.960710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:11.968873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1296/2552] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:11.973632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:11.983407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11e45-6be2-4be2-b8c6-fffa769d9dc1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:11.988508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:11.989102 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:11.989401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:11.989674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:11.990343 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:11.990648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:11.992334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.005395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1257/2553] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:12.010216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.020450 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446417-4758-47ea-a05e-e076608a0d64 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.025128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.025724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.026002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.026271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.026900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.028870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.064569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.067155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:12.127626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3d552-4979-437e-8379-253321aecdc9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.132050 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.132684 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.132969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.133248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.137285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:12.166647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1297/2554] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:12 2022] POST /identity/v3/projects => generated 359 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:12.183134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.201741 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:12.212022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.213504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.223350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1258/2555] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:12.224190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.232261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be1896-749d-411a-8bdd-8ce4bf583fda service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.251801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e739c8-807c-4e18-855e-7e44cb998a3c admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.258311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.258930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.259210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.259483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.260116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.262257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.290149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.291143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.291143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.291439 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.293240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:12.307781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.309262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.320721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1259/2556] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:12.440843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1298/2557] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:12 2022] POST /identity/v3/users => generated 280 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:12.444898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.455485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c56c4a-2d06-4179-b1ce-578e4c40d4b8 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.461295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.461909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.462193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.462471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.463119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.464767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.476386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1260/2558] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:12.480354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.490660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746aa9f-db11-4916-b980-bbc2da3d123a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.495301 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.495905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.496207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.496480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.497410 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.499003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dbaf3-0f8d-46be-810a-09eb9df791a0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.509769 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1299/2559] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:12.513983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.524649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf457c7-74e0-4bee-9801-4292d56ef76f admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.529331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.529981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.530263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.530540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] PATH_INFO: `/v3/projects/e2dca77321b4421db8bcbcbfe737b34e/users/2fb908e3c68a4cab8429826c1e606662/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.547047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2dca77321b4421db8bcbcbfe737b34e, user_id=2fb908e3c68a4cab8429826c1e606662, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.549806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.702219 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1261/2560] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:12 2022] PUT /identity/v3/projects/e2dca77321b4421db8bcbcbfe737b34e/users/2fb908e3c68a4cab8429826c1e606662/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:12.713374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.713752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.714096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.742977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] MFA Rules not processed for user `2fb908e3c68a4cab8429826c1e606662`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:12.744220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:12.811566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1300/2561] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:12.850414 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:12.851768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:12.867631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:12.916848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee97de-3a04-4332-963c-d8997e7cf0e2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:12.920986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:12.921592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:12.921873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:12.922143 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:12.936408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:12.937882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad7cd4-88e1-42e4-bd61-37fe10c8b62a service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:12.947554 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1262/2562] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:12.964340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:13.029112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93f62-8428-484d-8fd8-dde2953679f2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:13.033829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:13.035500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:13.035896 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:13.036226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] PATH_INFO: `/v3/users/2fb908e3c68a4cab8429826c1e606662` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:13.037382 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb908e3c68a4cab8429826c1e606662)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:13.039309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:13.182552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:13.288605 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb908e3c68a4cab8429826c1e606662'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:13.289557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb908e3c68a4cab8429826c1e606662'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:13.297273 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb908e3c68a4cab8429826c1e606662'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:13.404898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb908e3c68a4cab8429826c1e606662'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:13.405512 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb908e3c68a4cab8429826c1e606662'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:13.424174 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1301/2563] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:12 2022] DELETE /identity/v3/users/2fb908e3c68a4cab8429826c1e606662 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:13.428716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:13.429010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:13.429285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:13.449944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:13.451162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:13.512944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1263/2564] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:13.524180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:13.525857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:13.585131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7ceb2-a582-4cc8-9681-b7e536ac8b5f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:13.588781 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:13.589442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:13.589720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:13.589993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:13.599219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:13.600697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437936c8-fcd0-43ec-85fa-21cebb1f3d49 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:13.610210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1302/2565] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:14.286287 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:14.289535 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:14.313315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd377487-440a-4824-a6f5-cb5a6de0ce8a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:14.318747 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:14.319583 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:14.320043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:14.320953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] PATH_INFO: `/v3/projects/e2dca77321b4421db8bcbcbfe737b34e` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:14.321630 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2dca77321b4421db8bcbcbfe737b34e)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:14.324216 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:14.542667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2dca77321b4421db8bcbcbfe737b34e', 'request_id': 'req-67652c74-7d1f-47a1-ba45-c69638bddc7f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:14.549878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2dca77321b4421db8bcbcbfe737b34e', 'request_id': 'req-67652c74-7d1f-47a1-ba45-c69638bddc7f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:14.696556 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2dca77321b4421db8bcbcbfe737b34e', 'request_id': 'req-67652c74-7d1f-47a1-ba45-c69638bddc7f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:14.713519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] The token cache is being invalidate because project e2dca77321b4421db8bcbcbfe737b34e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:14.713814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:14.714474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-67652c74-7d1f-47a1-ba45-c69638bddc7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:14.716269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1264/2566] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:14 2022] DELETE /identity/v3/projects/e2dca77321b4421db8bcbcbfe737b34e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:14.720953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:14.722107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:14.722201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:14.747524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:14.748815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:14.876364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1303/2567] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:14.885220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:14.885504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:14.885781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:14.910625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:14.911947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:14.936552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1265/2568] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:14.941179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:14.957038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda44ef6-d49d-4787-b290-b9dce96aa3d6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:14.962820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:14.963575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:14.963955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:14.964359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:14.965128 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:14.965677 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:14.967569 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:14.983674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1304/2569] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:14:14.988125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:14.998187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6331-44ed-4a36-ac99-a12ee980b9e7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.004264 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.004891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.005174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.005451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.006095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.008122 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.151401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1266/2570] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:15.155753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.166029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2461-7a84-4688-bb71-24a41ea69471 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.203600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:15.234104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.234740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.235017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.235289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.236043 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.239415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.381136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1305/2571] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:15 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:15.385248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.392988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:15.397776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2cf8-5010-45be-9294-4542d824d639 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.406472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.407207 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.407543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.411184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.411184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.411184 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.425461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1267/2572] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:15.427156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.437943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48d309-a00d-40cd-8aaa-792b5cdfa245 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.444668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.445363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.445712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.446054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.446766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.448830 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc690f-5722-4ef2-98c5-89b2556b3205 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.466973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1306/2573] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:15.470049 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.481631 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de48e95-41ad-4fd7-b72c-f44f63bfe7ca admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.486634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.487857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.488171 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.488475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] PATH_INFO: `/v3/projects/a85c39c0710c49479669e0637e6db127/users/6e6c0f3f883744fbbf487722d598e6a3/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.505008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85c39c0710c49479669e0637e6db127, user_id=6e6c0f3f883744fbbf487722d598e6a3, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.507944 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.622541 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1268/2574] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:15 2022] PUT /identity/v3/projects/a85c39c0710c49479669e0637e6db127/users/6e6c0f3f883744fbbf487722d598e6a3/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:15.633866 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.634148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.634425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.693946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] MFA Rules not processed for user `6e6c0f3f883744fbbf487722d598e6a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:15.695363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:15.772693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1307/2575] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:14:15.778580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.780768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:15.786854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:15.794361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:15.891754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbab5c0-eac9-46ee-9af8-6e6e5e1e4aa2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.897507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.899256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.899737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.900028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.910465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.911953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.928336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1308/2576] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:15.934567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a90b-6ee2-47c0-9951-d78fce4d6d72 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:15.940937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:15.941571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:15.941854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:15.942132 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:15.944023 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:15.965096 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:15.966800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:15.980650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1269/2577] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:17.586660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:17.588258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:17.603828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba0d23-8eb9-4d7b-b190-75544eaa0d1a service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:17.609059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:17.609682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:17.610248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:17.610520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:17.612345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:17.625401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:17.627081 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:17.637771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1309/2578] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:17.743004 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:17.744551 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:17.810607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e78d3f-6384-4b0a-b41c-790ce2bb2386 service placement] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:17.814942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:17.816265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:17.816552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:17.816822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] PATH_INFO: `/v3/users/fe93db6503a143ab9ed50fedbc8f010f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:17.817922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe93db6503a143ab9ed50fedbc8f010f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:17.819864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:18.025470 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:18.030186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe93db6503a143ab9ed50fedbc8f010f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.030841 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe93db6503a143ab9ed50fedbc8f010f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.037872 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe93db6503a143ab9ed50fedbc8f010f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.130283 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe93db6503a143ab9ed50fedbc8f010f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.131166 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-09b1cb9c-2823-48a6-afda-2d70d2997019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe93db6503a143ab9ed50fedbc8f010f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.159835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1270/2579] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:17 2022] DELETE /identity/v3/users/fe93db6503a143ab9ed50fedbc8f010f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:18.578655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:18.587626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2262f9-a00f-4c8c-b237-c6bb30192e81 service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:18.622647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:18.652717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:18.655156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:18.655156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:18.655156 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] PATH_INFO: `/v3/projects/1d3d0ec7db3c4b3b8625c575a2a11ec8` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:18.655449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3d0ec7db3c4b3b8625c575a2a11ec8)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:18.657290 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:18.856244 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3d0ec7db3c4b3b8625c575a2a11ec8', 'request_id': 'req-9a8a1838-c21c-4aaa-8146-1770ef5447f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.861417 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3d0ec7db3c4b3b8625c575a2a11ec8', 'request_id': 'req-9a8a1838-c21c-4aaa-8146-1770ef5447f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.869293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3d0ec7db3c4b3b8625c575a2a11ec8', 'request_id': 'req-9a8a1838-c21c-4aaa-8146-1770ef5447f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.969784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] The token cache is being invalidate because project 1d3d0ec7db3c4b3b8625c575a2a11ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:18.970115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.975930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9a8a1838-c21c-4aaa-8146-1770ef5447f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:18.975930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1310/2580] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:18 2022] DELETE /identity/v3/projects/1d3d0ec7db3c4b3b8625c575a2a11ec8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:18.977518 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:18.978039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:18.978320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:18.998806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:19.000088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.060833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1271/2581] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:19.069235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.069529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.069811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.090365 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:19.091582 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.099770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1311/2582] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:19.104201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.114810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2674fc-0094-4dd2-a8bb-6d42fc43b561 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.119311 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.119932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.120234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.120536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.121190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:19.121589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.123514 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.136341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1272/2583] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:14:19.140844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.150744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91f182-9275-4f4f-94bb-1471129ef7d9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.155754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.156428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.156711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.157018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.158648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.161082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.271576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1312/2584] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:19 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:19.282413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.294288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3e3f-3f1e-4835-aef1-84c2a73bfdad admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.355028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.355634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.355909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.356203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.356863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.358833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.375076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.376812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.389431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7434fea-e861-4fd5-a8b3-5a31aba92895 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.394342 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.394659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.394939 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.395212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] PATH_INFO: `/v3/users/6e6c0f3f883744fbbf487722d598e6a3` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.396357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6c0f3f883744fbbf487722d598e6a3)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.398387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.462255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1273/2585] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:19 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:19.466274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.476217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc5a68-3ade-4898-a65e-5c0cf3d9451d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.480793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.481364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.481637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.481912 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.482542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.484160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.495100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1274/2586] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:19.499115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.508483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221985-392b-46ce-a152-4baed2eefc54 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.513099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.513657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.513935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.514206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.514813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.516398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.525914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1275/2587] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:19.529740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.539952 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d1933-62ec-40c9-ba5e-fe8880a22473 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.544795 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.549193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.549193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.549193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] PATH_INFO: `/v3/projects/2a5f3f3c8da84b49850bee8c32d1ce94/users/d86706c2feb840cb9cc85b530be372f0/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.557668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:19.561462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5f3f3c8da84b49850bee8c32d1ce94, user_id=d86706c2feb840cb9cc85b530be372f0, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.564129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6ac8f-922b-4437-84ce-cedc05c6cc63 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.630247 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6c0f3f883744fbbf487722d598e6a3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:19.630849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6c0f3f883744fbbf487722d598e6a3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:19.637919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6c0f3f883744fbbf487722d598e6a3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:19.690549 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1276/2588] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:19 2022] PUT /identity/v3/projects/2a5f3f3c8da84b49850bee8c32d1ce94/users/d86706c2feb840cb9cc85b530be372f0/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:19.701634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.701921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.702195 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.718840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6c0f3f883744fbbf487722d598e6a3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:19.719460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-21c5ed63-c8b1-4308-ba0a-689e3e07eba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6c0f3f883744fbbf487722d598e6a3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:19.738386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1313/2589] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:19 2022] DELETE /identity/v3/users/6e6c0f3f883744fbbf487722d598e6a3 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:19.742774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.743058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.743330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.746792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] MFA Rules not processed for user `d86706c2feb840cb9cc85b530be372f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:19.748616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.752188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:19.763829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:19.765111 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.818226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1277/2590] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:14:19.825501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1314/2591] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:14:19.833564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.834894 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.839487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:19.840821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:19.901735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179d9a-1239-4713-89e5-3bf2212279a0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.905297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.906116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.906396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.906676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.916742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.918179 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.929086 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1315/2592] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:19.944293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316d62f-c642-48da-b356-aa80a8fcfe0c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:19.949968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:19.950579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:19.950857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:19.951127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:19.961600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:19.962375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a1e61-ba6b-4021-ae66-3d9074993bf5 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:19.971762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1278/2593] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:20.458577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:20.460177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:20.474959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154fdf0-e164-4b85-8f06-7a88c552c8d8 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:20.480831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:20.481523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.481851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.482131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] PATH_INFO: `/v3/projects/a85c39c0710c49479669e0637e6db127` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.483560 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85c39c0710c49479669e0637e6db127)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:20.486096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:20.648346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85c39c0710c49479669e0637e6db127', 'request_id': 'req-28edf2b1-2c8d-407e-8d96-87d48672158b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:20.654824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85c39c0710c49479669e0637e6db127', 'request_id': 'req-28edf2b1-2c8d-407e-8d96-87d48672158b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:20.662882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85c39c0710c49479669e0637e6db127', 'request_id': 'req-28edf2b1-2c8d-407e-8d96-87d48672158b'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:20.762603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] The token cache is being invalidate because project a85c39c0710c49479669e0637e6db127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:20.762603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:20.769198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:20.771652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1316/2594] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:20 2022] DELETE /identity/v3/projects/a85c39c0710c49479669e0637e6db127 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:20.776463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb838-d071-459b-b207-1a395d62bde0 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.776942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb838-d071-459b-b207-1a395d62bde0 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.777042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb838-d071-459b-b207-1a395d62bde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.788350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:20.800036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:20.805375 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-bd3eb838-d071-459b-b207-1a395d62bde0 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:20.806574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bd3eb838-d071-459b-b207-1a395d62bde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:20.844195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:20.848600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:20.873838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edf2b1-2c8d-407e-8d96-87d48672158b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:20.877606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:20.878217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.878675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.879061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.881360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:20.884559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1279/2595] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:20.892223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.892699 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.892981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.895883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:20.897578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:20.910588 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1317/2596] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:20.917649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:20.917649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:20.929905 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1280/2597] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:20.932837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:20.943324 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e7c9a-b6b2-4145-9db4-19267a5b5b02 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:20.948068 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:20.948839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.949012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.949257 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.949943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:20.950195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:20.951790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:20.974308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1318/2598] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:20.978580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:20.989149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29b62-505c-4154-b1f7-ca8051719bf1 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:20.994145 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:20.994750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:20.995026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:20.995296 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:20.995938 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:20.998230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.166653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1281/2599] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:20 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:21.171020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:21.182595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704989cf-5f7a-49d6-a03b-c10d51e4e251 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:21.241439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:21.242054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.242337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.242802 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.243470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:21.245444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.416754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1319/2600] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:21 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:21.420830 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:21.432417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d113f1-49e8-4ea0-89c9-2ace638a3183 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:21.437114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:21.438215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.438408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.438683 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.439328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:21.441585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.451739 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1282/2601] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:21.455950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:21.466057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b440-7248-490c-b066-b47ba94a7638 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:21.470811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:21.471415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.471691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.471965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.472748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:21.474336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37374e4-29ae-4f3c-aca1-50431f96374c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.484785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1320/2602] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:21.489856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:21.500341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39313f2b-6d35-4442-b216-89b3632bd4b6 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:21.505598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:21.506259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.506539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.506812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] PATH_INFO: `/v3/projects/b6c389c35c7f433a912a8c1dbde72f69/users/7b8fe8d5a5d84183a515eb49ec75a21c/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.522783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c389c35c7f433a912a8c1dbde72f69, user_id=7b8fe8d5a5d84183a515eb49ec75a21c, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:21.525561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.628274 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1283/2603] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:21 2022] PUT /identity/v3/projects/b6c389c35c7f433a912a8c1dbde72f69/users/7b8fe8d5a5d84183a515eb49ec75a21c/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:21.638631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2504-baf8-4c42-b8a6-de559e45e2bc None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.638912 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2504-baf8-4c42-b8a6-de559e45e2bc None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.639196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2504-baf8-4c42-b8a6-de559e45e2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.667980 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f94f2504-baf8-4c42-b8a6-de559e45e2bc None None] MFA Rules not processed for user `7b8fe8d5a5d84183a515eb49ec75a21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:21.669284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f94f2504-baf8-4c42-b8a6-de559e45e2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:21.737127 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1321/2604] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:21.748978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:21.750889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:21.811100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc92c7-3226-499f-b93d-d93d514a9139 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:21.815333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:21.816240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:21.816624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:21.817126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:21.828055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:21.829564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:21.838547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1284/2605] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:26.210462 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:26.211333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:26.211790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:26.232480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] MFA Rules not processed for user `7b8fe8d5a5d84183a515eb49ec75a21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:26.234811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:26.241757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1322/2606] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:28.985241 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:28.986771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:29.060955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5092-738b-494e-bdd0-fda16668d07b service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:29.061062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:29.061700 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:29.061986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:29.062273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] PATH_INFO: `/v3/users/2927c54860d840538bab1004441c104b` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:29.063614 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2927c54860d840538bab1004441c104b)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:29.065698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:29.090825 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:29.321491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2927c54860d840538bab1004441c104b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:29.322157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2927c54860d840538bab1004441c104b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:29.329082 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2927c54860d840538bab1004441c104b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:29.495738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2927c54860d840538bab1004441c104b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:29.496388 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2927c54860d840538bab1004441c104b'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:29.515014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1285/2607] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:28 2022] DELETE /identity/v3/users/2927c54860d840538bab1004441c104b => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:29.777192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:29.798832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d871cb8-c3ef-4b96-a7ee-3139f3055d30 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:29.803069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:29.803069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:29.826069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:29.866723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:29.867469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:29.867822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:29.868104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] PATH_INFO: `/v3/projects/54df2648916c4681af69062a1d5b7430` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:29.869267 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54df2648916c4681af69062a1d5b7430)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:29.871166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:29.882152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557b5b6-7ca2-44a7-bb68-5ec378ad7d96 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:29.886760 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:29.887474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:29.887761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:29.888034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] PATH_INFO: `/v3/users/d86706c2feb840cb9cc85b530be372f0` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:29.889180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86706c2feb840cb9cc85b530be372f0)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:29.891086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:30.009379 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54df2648916c4681af69062a1d5b7430', 'request_id': 'req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.015056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54df2648916c4681af69062a1d5b7430', 'request_id': 'req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.021660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54df2648916c4681af69062a1d5b7430', 'request_id': 'req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.118454 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] The token cache is being invalidate because project 54df2648916c4681af69062a1d5b7430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:30.118750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.119486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.121048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1323/2608] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:29 2022] DELETE /identity/v3/projects/54df2648916c4681af69062a1d5b7430 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:30.125161 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:30.126602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:30.190494 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86706c2feb840cb9cc85b530be372f0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.191281 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86706c2feb840cb9cc85b530be372f0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.201806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d9f4-3b53-4250-b3d2-cd8d0fe63938 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:30.207619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:30.208298 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:30.208586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:30.208849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] PATH_INFO: `/v3/users/146e0804900a41598c1fd5b4508d7b88` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:30.210010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146e0804900a41598c1fd5b4508d7b88)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:30.212074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86706c2feb840cb9cc85b530be372f0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.212337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:30.324796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86706c2feb840cb9cc85b530be372f0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.325922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-964b4684-c0e1-4338-9199-e3db22d37e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86706c2feb840cb9cc85b530be372f0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.347436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1286/2609] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:29 2022] DELETE /identity/v3/users/d86706c2feb840cb9cc85b530be372f0 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:30.352080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:30.352474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:30.352987 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:30.374243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:30.375619 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:30.385240 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:30.435391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1287/2610] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:30.445363 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:30.446780 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:30.448542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146e0804900a41598c1fd5b4508d7b88'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.449189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146e0804900a41598c1fd5b4508d7b88'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.454642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:30.457310 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146e0804900a41598c1fd5b4508d7b88'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.528859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570abf4c-1e5d-42df-954d-c24dcc79996a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:30.532661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:30.533301 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:30.533639 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:30.533856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:30.544347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:30.545817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:30.556447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1288/2611] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:14:30.598668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146e0804900a41598c1fd5b4508d7b88'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.599414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146e0804900a41598c1fd5b4508d7b88'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:30.618678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1324/2612] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:30 2022] DELETE /identity/v3/users/146e0804900a41598c1fd5b4508d7b88 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:30.947279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:30.947279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:30.963369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5f273-f3b8-43c9-ab1b-8f832b853b7d service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:31.025403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:31.026035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:31.026316 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:31.026594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] PATH_INFO: `/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:31.027730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbfe6b887aa041b7a0de4ce722cb1573)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:31.029669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:31.188936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:31.195731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfe6b887aa041b7a0de4ce722cb1573', 'request_id': 'req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.201279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfe6b887aa041b7a0de4ce722cb1573', 'request_id': 'req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.290066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfe6b887aa041b7a0de4ce722cb1573', 'request_id': 'req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.306209 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] The token cache is being invalidate because project dbfe6b887aa041b7a0de4ce722cb1573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:31.306480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.307155 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-04d1f3ea-3cfc-498c-ab79-b0bf048f437f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.309036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1289/2613] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:30 2022] DELETE /identity/v3/projects/dbfe6b887aa041b7a0de4ce722cb1573 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:31.623311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:31.624891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:31.649047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3309-f0e5-47bd-974a-e061a91842a5 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:31.719221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:31.719860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:31.720139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:31.720451 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] PATH_INFO: `/v3/projects/2a5f3f3c8da84b49850bee8c32d1ce94` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:31.721587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5f3f3c8da84b49850bee8c32d1ce94)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:31.724360 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:31.839806 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5f3f3c8da84b49850bee8c32d1ce94', 'request_id': 'req-c03a1fc6-7c90-4c75-938d-7c05d8640400'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.845943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5f3f3c8da84b49850bee8c32d1ce94', 'request_id': 'req-c03a1fc6-7c90-4c75-938d-7c05d8640400'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.853647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5f3f3c8da84b49850bee8c32d1ce94', 'request_id': 'req-c03a1fc6-7c90-4c75-938d-7c05d8640400'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.992331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] The token cache is being invalidate because project 2a5f3f3c8da84b49850bee8c32d1ce94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:31.992622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.993253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c03a1fc6-7c90-4c75-938d-7c05d8640400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:31.994811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1325/2614] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:31 2022] DELETE /identity/v3/projects/2a5f3f3c8da84b49850bee8c32d1ce94 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:32.000862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.001172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.001447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.021984 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:32.023167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:32.086704 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1290/2615] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:14:32.094407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.094690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.094963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.117688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:32.118925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:32.127233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1326/2616] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:32.132663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.142486 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e818-90a2-4415-922b-fbe3fcb5275d None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.148262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.148869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.149212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.149441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.150083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:32.150387 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.152320 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.164860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1291/2617] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:32.169369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.178997 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc324-f570-4ec8-bb22-f328da4575ea None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.183891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.184527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.184905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.185189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.185811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.187715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.332822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1327/2618] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:32 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:32.336859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.347121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e03972-cce9-45e9-b1a0-38ca3146fae6 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.412888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.413540 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.413817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.414085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.414722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.416882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.533069 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1292/2619] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:32 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:32.534555 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.544618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f9bd9-4d3d-47a1-8247-564cf5927c46 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.549255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.550591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.551026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.551361 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.552070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.554220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.564400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1328/2620] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:32.569400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.582464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a6acc-d4fc-4162-bae0-333aa898fc49 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.587476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.588259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.588741 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.589112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.589861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.596433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf01ff-f6f8-4c69-b7ac-b06b576d47a1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.614847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1293/2621] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:32.618968 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.630702 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3adadd-aeb8-4181-bcd4-ccc94661978a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.635551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.636243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.636520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.636989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] PATH_INFO: `/v3/projects/fe5c0009d8ac49e182128b388737f5e6/users/e5acae39b1b24161b97b03154bfd2271/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.653330 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c0009d8ac49e182128b388737f5e6, user_id=e5acae39b1b24161b97b03154bfd2271, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.655973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:32.782023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:32.788297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1329/2622] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:32 2022] PUT /identity/v3/projects/fe5c0009d8ac49e182128b388737f5e6/users/e5acae39b1b24161b97b03154bfd2271/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:32.798507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.798837 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.807304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.827769 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] MFA Rules not processed for user `e5acae39b1b24161b97b03154bfd2271`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:32.829012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:32.855319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:32.895980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1294/2623] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:32.900574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:32.964659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3230fd-29e4-4d17-bea8-31eb9db220da admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:32.969420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:32.970079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:32.970359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:32.970630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:32.971262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:32.973954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.086762 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1330/2624] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:32 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:33.090759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.105298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156d984-9f94-431f-a4a9-061fa609ca67 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.162866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.163477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.163762 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.164038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.164725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.166655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.284391 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1295/2625] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:33 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:33.288858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.299005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780da9b-e582-49d5-9412-83705baf5816 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.305857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.306463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.306742 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.307011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.307649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.309276 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.321282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1331/2626] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:33.326896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.337215 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd09-6e50-4f6d-8acb-d683951a4a32 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.341783 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.342434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.342717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.343015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] PATH_INFO: `/v3/projects/aea4c964ae644462b84fb32d7dbc1832/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.357721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea4c964ae644462b84fb32d7dbc1832, user_id=d5bcea3f9a3f4ec1bcc2f5dc67da1fda, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.360416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.432581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1296/2627] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:33 2022] PUT /identity/v3/projects/aea4c964ae644462b84fb32d7dbc1832/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:33.436484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.447526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251cdfd-1768-426b-b4e2-ba1fbaa333df admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.503543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.504695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.504978 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.505249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.505889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.507480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e90a5-5ae6-4cca-a415-cd4a4068ff5d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.517039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1332/2628] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:33.521323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.531722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3997-a894-44d2-b2f6-f85ffca6b87d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.536874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.537453 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.537732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.538001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] PATH_INFO: `/v3/projects/aea4c964ae644462b84fb32d7dbc1832/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.539890 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea4c964ae644462b84fb32d7dbc1832, user_id=d5bcea3f9a3f4ec1bcc2f5dc67da1fda, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.542520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.553149 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1297/2629] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:33 2022] PUT /identity/v3/projects/aea4c964ae644462b84fb32d7dbc1832/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:33.564746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.565059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.565336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.592648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] MFA Rules not processed for user `d5bcea3f9a3f4ec1bcc2f5dc67da1fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:33.593857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:33.653733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:33.664286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1333/2630] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:33.677390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:33.679160 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:33.757374 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61946a-b744-43ca-a53d-b248a805bacd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:33.761192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:33.761789 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:33.762085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:33.762319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:33.771629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:33.773092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:33.778418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:33.783228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1298/2631] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:34.988948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:34.998368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6e931-5057-4b00-ad17-77d2a473f347 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:35.002965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aea4c964ae644462b84fb32d7dbc1832', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:35.003836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:35.003836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:35.004090 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:35.004770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:35.005074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:35.006676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:35.018286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1334/2632] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:35.023224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:35.032696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2b76-68a0-4b7e-92b5-8def0b4efc80 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:35.038662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aea4c964ae644462b84fb32d7dbc1832', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:35.039401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:35.039743 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:35.040083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:35.040821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:35.042801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:35.142042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1299/2633] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:35 2022] POST /identity/v3/projects => generated 354 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:35.146906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:35.156865 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73f0d-8c4c-4114-a13c-8a6a34eaf0c2 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:35.211305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aea4c964ae644462b84fb32d7dbc1832', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:35.211902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:35.212223 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:35.212499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:35.213124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:35.213431 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:35.215077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:35.227467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1335/2634] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:35.264430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:35.275674 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017b031-5751-4098-ba0e-40adb4a5edbe tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:35.280263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aea4c964ae644462b84fb32d7dbc1832', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:35.280886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:35.281169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:35.281444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] PATH_INFO: `/v3/projects/c66dd39d76b74a5abb0d5f2a3f7531d5` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:35.282578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c66dd39d76b74a5abb0d5f2a3f7531d5)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:35.284667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:35.417205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66dd39d76b74a5abb0d5f2a3f7531d5', 'request_id': 'req-631161df-3aaf-4d9d-b210-dbbf514937b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:35.432043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66dd39d76b74a5abb0d5f2a3f7531d5', 'request_id': 'req-631161df-3aaf-4d9d-b210-dbbf514937b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:35.552164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66dd39d76b74a5abb0d5f2a3f7531d5', 'request_id': 'req-631161df-3aaf-4d9d-b210-dbbf514937b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:35.563770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] The token cache is being invalidate because project c66dd39d76b74a5abb0d5f2a3f7531d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:35.569568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:35.569568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-631161df-3aaf-4d9d-b210-dbbf514937b0 tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:35.569568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1300/2635] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:35 2022] DELETE /identity/v3/projects/c66dd39d76b74a5abb0d5f2a3f7531d5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:36.482459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:36.484251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:36.552432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80148da1-94bb-4dea-b67f-458183fec96f tempest-VolumeQuotaClassesTest-340634250 tempest-VolumeQuotaClassesTest-340634250-project-admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:36.731874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:36.732468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:36.732747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:36.733026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] PATH_INFO: `/v3/users/e5acae39b1b24161b97b03154bfd2271` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:36.734155 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5acae39b1b24161b97b03154bfd2271)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:36.736063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:36.759747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:36.949339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5acae39b1b24161b97b03154bfd2271'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:36.950023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5acae39b1b24161b97b03154bfd2271'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:36.956914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5acae39b1b24161b97b03154bfd2271'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:37.072094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5acae39b1b24161b97b03154bfd2271'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:37.072779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5acae39b1b24161b97b03154bfd2271'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:37.090388 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1336/2636] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:36 2022] DELETE /identity/v3/users/e5acae39b1b24161b97b03154bfd2271 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:37.095167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:37.095667 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:37.095775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:37.116305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:37.118298 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:37.177522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1301/2637] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:37.187323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:37.188779 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:37.244953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bc4ce-7b3c-4cc1-b9f3-456a1c10046d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:37.248750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:37.249349 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:37.249622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:37.249892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:37.259546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:37.260966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:37.270478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1337/2638] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:37.754134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:37.755649 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:37.772965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2477bf-ab6a-469f-a5b3-765824f20b95 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:37.777500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:37.778130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:37.778428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:37.778665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] PATH_INFO: `/v3/projects/fe5c0009d8ac49e182128b388737f5e6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:37.779980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5c0009d8ac49e182128b388737f5e6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:37.781908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:37.977208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5c0009d8ac49e182128b388737f5e6', 'request_id': 'req-822ad95d-642e-404a-a6fa-d9e123b22713'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:37.982013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:37.983454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5c0009d8ac49e182128b388737f5e6', 'request_id': 'req-822ad95d-642e-404a-a6fa-d9e123b22713'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.083919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5c0009d8ac49e182128b388737f5e6', 'request_id': 'req-822ad95d-642e-404a-a6fa-d9e123b22713'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.095323 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] The token cache is being invalidate because project fe5c0009d8ac49e182128b388737f5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:38.095919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.096258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.097945 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1302/2639] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:37 2022] DELETE /identity/v3/projects/fe5c0009d8ac49e182128b388737f5e6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:38.102054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:38.111599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32ffb-8ab1-4058-8526-0a86fe9c8ecd service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:38.170776 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:38.171857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:38.172398 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:38.172867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] PATH_INFO: `/v3/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:38.174841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5bcea3f9a3f4ec1bcc2f5dc67da1fda)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:38.177372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:38.650973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:38.652576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.653255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.660278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.763770 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.764426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-6a113950-fd16-4d4a-a280-9b2487e1b8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5bcea3f9a3f4ec1bcc2f5dc67da1fda'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:38.783123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1338/2640] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:38 2022] DELETE /identity/v3/users/d5bcea3f9a3f4ec1bcc2f5dc67da1fda => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:39.284425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:39.286070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:39.300072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ad95d-642e-404a-a6fa-d9e123b22713 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:39.357237 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:39.357867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:39.358150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:39.358424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] PATH_INFO: `/v3/projects/aea4c964ae644462b84fb32d7dbc1832` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:39.359537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea4c964ae644462b84fb32d7dbc1832)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:39.361483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:39.519807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea4c964ae644462b84fb32d7dbc1832', 'request_id': 'req-abe11e94-6276-4433-99e5-972c8f227cc0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:39.525605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea4c964ae644462b84fb32d7dbc1832', 'request_id': 'req-abe11e94-6276-4433-99e5-972c8f227cc0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:39.638064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea4c964ae644462b84fb32d7dbc1832', 'request_id': 'req-abe11e94-6276-4433-99e5-972c8f227cc0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:39.650881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] The token cache is being invalidate because project aea4c964ae644462b84fb32d7dbc1832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:39.651168 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:39.651796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-abe11e94-6276-4433-99e5-972c8f227cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:39.653655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1303/2641] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:39 2022] DELETE /identity/v3/projects/aea4c964ae644462b84fb32d7dbc1832 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:39.658631 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:39.659134 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:39.659756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:39.679703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:39.680958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:39.740670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1339/2642] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:14:39.748835 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:39.749126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:39.749410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:39.770953 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:39.773133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:39.781495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1304/2643] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:39.786724 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:39.796670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e126817-08ea-4435-b0a9-9c3e6cf5633d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:39.801446 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:39.802052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:39.802403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:39.802679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:39.803329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:39.803627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:39.805752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:39.819176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1340/2644] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:39.823613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:39.833807 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782acde-11c8-4919-9591-70bc03a81085 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:39.838653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:39.839261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:39.839768 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:39.840227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:39.840860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:39.842831 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:39.936696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1305/2645] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:39 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:39.940842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:39.950940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d2995-b2c9-4f76-9365-ee72703f755e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.008033 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.008683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.009003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.009296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.009952 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.011829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.129430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1341/2646] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:39 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:40.133663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.145024 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2accefd-5503-4859-ab59-9aa22bde0b26 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.149627 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.152610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.152893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.155328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.158218 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.160761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.170702 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1306/2647] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:40.174817 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.184845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e113d-e992-4dd9-bfe6-ee88d1483eb9 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.189626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.190216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.190528 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.190796 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.191422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.193148 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18965c9-c864-4ce0-8811-9cfbe78b711c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.203374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1342/2648] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:40.208062 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.218152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e04c7-cfbd-49bc-b95d-4bf72815614e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.223641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.224329 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.224612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.224908 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] PATH_INFO: `/v3/projects/5b84351a3a1d406ca68ed3c74d25ec6c/users/78b1b9ed03c54d54b72ab94b747bc363/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.239545 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b84351a3a1d406ca68ed3c74d25ec6c, user_id=78b1b9ed03c54d54b72ab94b747bc363, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.243705 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.336931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1307/2649] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:40 2022] PUT /identity/v3/projects/5b84351a3a1d406ca68ed3c74d25ec6c/users/78b1b9ed03c54d54b72ab94b747bc363/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:40.347384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.347659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.347932 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.377359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] MFA Rules not processed for user `78b1b9ed03c54d54b72ab94b747bc363`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:40.379520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:40.397863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:40.450409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1343/2650] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:40.455081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.470295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:40.521425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bbefe-48b6-4ec0-a628-ca1e9197b759 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.526211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.533712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.533712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.533712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.533712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.533712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.685385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1308/2651] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:40 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:40.689820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.701318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279de809-0d4c-43e5-9d68-c349252765b9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.756620 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.757221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.757495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.757764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.758387 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.760285 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:40.973516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1344/2652] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:40 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:14:40.974770 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:40.985159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93c3b-d973-4900-8bac-588e69855c43 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:40.990800 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:40.991452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:40.991812 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:40.992089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:40.992790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:40.994398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.006425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1309/2653] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:41.010444 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.020836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c877e-9282-47e7-9c7b-865b2fe7a265 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.025668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.026270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.026549 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.026831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.027466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.029085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90cba-4c5b-42bd-a72d-be511f20b45d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.041518 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1345/2654] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:41.044190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.056711 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ccc4b-38d0-498a-9ab9-9b9f48fe04ea admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.064113 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.065078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.065465 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.065899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] PATH_INFO: `/v3/projects/b55c267a7174495fa7b81d7b11a3d00b/users/e39777d27ef74d0fbe6ac2702a490676/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.081591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55c267a7174495fa7b81d7b11a3d00b, user_id=e39777d27ef74d0fbe6ac2702a490676, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.084269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.209491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1310/2655] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:41 2022] PUT /identity/v3/projects/b55c267a7174495fa7b81d7b11a3d00b/users/e39777d27ef74d0fbe6ac2702a490676/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:41.220768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.221097 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.221967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.251139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] MFA Rules not processed for user `e39777d27ef74d0fbe6ac2702a490676`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:41.252318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:41.320625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1346/2656] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:41.325123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.388420 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3eeb4-15ac-40bd-b170-abcf8794b756 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.393976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.394605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.394878 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.395144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.395813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.398493 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.589968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1311/2657] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:41 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:41.593981 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.605085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205120b-c8d9-4376-b985-bf9e8f43ea50 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.631571 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:41.665260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.665876 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.666154 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.666428 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.667066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.668987 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.826851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1347/2658] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:41 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:41.831382 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.841765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178b655-db83-4e02-8dbb-a8309766db69 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.846412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.847183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.847472 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.847750 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.848454 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.850061 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:41.860568 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1312/2659] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:41.864958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:41.875826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ecd11-fb43-4183-8f43-0a124b3a0161 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:41.880915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:41.881564 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:41.881857 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:41.882137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] PATH_INFO: `/v3/projects/8944029deb024cb3839a55878a10ffbf/users/616d2de6b5324f57aea5c6faadbac327/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:41.900828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8944029deb024cb3839a55878a10ffbf, user_id=616d2de6b5324f57aea5c6faadbac327, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:41.904052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:42.014824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1348/2660] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:41 2022] PUT /identity/v3/projects/8944029deb024cb3839a55878a10ffbf/users/616d2de6b5324f57aea5c6faadbac327/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:42.019653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:42.029814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9f93-7247-476f-828c-456f0f31bef0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:42.062815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:42.087427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:42.088040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:42.088356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:42.088634 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:42.089279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:42.090880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382082f-32e4-47cb-92fc-bd0bb361cb23 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:42.100447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1313/2661] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:42.104586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:42.115331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eedfb-6ef7-46bf-a58e-6dbeb7fa5cb3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:42.119908 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:42.120573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:42.120851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:42.121123 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] PATH_INFO: `/v3/projects/8944029deb024cb3839a55878a10ffbf/users/616d2de6b5324f57aea5c6faadbac327/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:42.123291 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8944029deb024cb3839a55878a10ffbf, user_id=616d2de6b5324f57aea5c6faadbac327, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:42.125900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:42.136541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1349/2662] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:42 2022] PUT /identity/v3/projects/8944029deb024cb3839a55878a10ffbf/users/616d2de6b5324f57aea5c6faadbac327/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:42.147190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:42.147480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:42.147758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:42.173969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] MFA Rules not processed for user `616d2de6b5324f57aea5c6faadbac327`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:42.175845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:42.242252 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1314/2663] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:42.254234 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:42.255935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:42.316466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdf161-e0fb-47bb-8736-f17e0398b67e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:42.320073 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:42.320704 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:42.320990 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:42.321275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:42.330681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:42.332517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef356d8-9ed8-4283-bab4-14623c6ad2d4 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:42.341873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1350/2664] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:43.618691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:43.620977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:43.636013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f951f-cc96-437b-bb49-78811c3d7f65 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:43.640969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:43.641824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:43.642144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:43.642417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:43.709610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:43.711107 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:43.721385 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1315/2665] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:46.719188 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d096-391c-4b3b-9f53-d7c758e4c8ef None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:46.720180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d096-391c-4b3b-9f53-d7c758e4c8ef None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:46.720486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d096-391c-4b3b-9f53-d7c758e4c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:46.755580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-a4d4d096-391c-4b3b-9f53-d7c758e4c8ef None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:46.756887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a4d4d096-391c-4b3b-9f53-d7c758e4c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:46.813052 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1351/2666] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:14:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:46.971058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:46.971058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:47.029380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=100 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:47.048657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70559-c958-4f21-a237-1abf9acc5f03 service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:47.054708 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:47.055328 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:47.056939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:47.056939 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:47.070012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:47.071903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:47.081016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1316/2667] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:14:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:14:48.052064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:48.053037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:48.053037 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:48.077524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:48.079258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:48.088537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1352/2668] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:48.102278 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:48.114883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f161-9da3-445d-b831-6b634f846b90 service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:48.121646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:48.122468 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:48.122740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:48.123007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:48.133517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:48.135342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38316f79-6b65-4633-aaff-20053c8478c1 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:48.143725 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1317/2669] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:14:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:14:49.128319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:49.142319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9d7cb-ccfd-44b2-ad40-7f73da37b5a5 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:49.146030 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:49.146667 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:49.146945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:49.147215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:49.157684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:49.206943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:49.208412 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:49.217471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1353/2670] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:49.251516 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cdfd7-e3be-47a6-b58d-e26665703041 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:49.252009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cdfd7-e3be-47a6-b58d-e26665703041 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:49.252128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cdfd7-e3be-47a6-b58d-e26665703041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:49.275091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-014cdfd7-e3be-47a6-b58d-e26665703041 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:49.276558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-014cdfd7-e3be-47a6-b58d-e26665703041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:49.285717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1318/2671] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:49.299959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:49.312901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233d60-d467-44b8-ad80-0994c19c4c6e service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:49.318396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:49.322798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:49.322798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:49.322798 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:49.328899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:49.330467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:49.339481 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1354/2672] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:14:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:14:50.124867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:50.124867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:50.124867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:50.143404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:50.145253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:50.152386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1319/2673] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:14:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:50.169078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:50.175271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953edf3-85df-4f8e-be9a-dee276f42ac8 service swift] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:50.180259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:50.180846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:50.181124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:50.181395 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:50.189347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:50.191436 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71904664-da52-4743-bfae-f4b8eb639b27 service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:50.199199 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1355/2674] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:14:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:14:53.331776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:53.334092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:53.399519 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d0338-4dd6-4d86-a1fb-ae3ce4c419d5 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:53.403978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:53.404625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:53.404921 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:53.406151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] PATH_INFO: `/v3/users/7b8fe8d5a5d84183a515eb49ec75a21c` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:53.406151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8fe8d5a5d84183a515eb49ec75a21c)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:53.408189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:53.640400 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8fe8d5a5d84183a515eb49ec75a21c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:53.641060 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8fe8d5a5d84183a515eb49ec75a21c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:53.647955 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8fe8d5a5d84183a515eb49ec75a21c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:53.740779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8fe8d5a5d84183a515eb49ec75a21c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:53.741544 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8fe8d5a5d84183a515eb49ec75a21c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:53.759839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1320/2675] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:53 2022] DELETE /identity/v3/users/7b8fe8d5a5d84183a515eb49ec75a21c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:53.764117 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:53.764615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:53.764738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:53.784160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:53.786014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:53.846683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1356/2676] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:53.857153 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:53.858592 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:53.901284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:53.917133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa075916-783f-498e-8660-01d223612a46 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:53.921048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:53.921661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:53.921941 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:53.922214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:53.932105 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:53.933600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:53.943534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1321/2677] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:54.241791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:54.243916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:54.261258 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566accd2-2815-4788-8d02-da656f07b6c6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:54.266053 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:54.275845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:54.275845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:54.275845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] PATH_INFO: `/v3/projects/b6c389c35c7f433a912a8c1dbde72f69` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:54.275845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c389c35c7f433a912a8c1dbde72f69)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:54.275845 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:54.284991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:54.287570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:54.305817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041d14-a05a-44f1-ad4b-d0b86b1611e4 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:54.317224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:54.317935 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:54.318294 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:54.318629 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] PATH_INFO: `/v3/users/78b1b9ed03c54d54b72ab94b747bc363` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:54.319924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b1b9ed03c54d54b72ab94b747bc363)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:54.322058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:54.438112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c389c35c7f433a912a8c1dbde72f69', 'request_id': 'req-45274578-99df-4f2a-ab4d-16d533f5d624'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.443831 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c389c35c7f433a912a8c1dbde72f69', 'request_id': 'req-45274578-99df-4f2a-ab4d-16d533f5d624'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.449945 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c389c35c7f433a912a8c1dbde72f69', 'request_id': 'req-45274578-99df-4f2a-ab4d-16d533f5d624'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.571384 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] The token cache is being invalidate because project b6c389c35c7f433a912a8c1dbde72f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:54.571596 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.572292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-45274578-99df-4f2a-ab4d-16d533f5d624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.573942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1357/2678] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:54 2022] DELETE /identity/v3/projects/b6c389c35c7f433a912a8c1dbde72f69 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:54.605230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b1b9ed03c54d54b72ab94b747bc363'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.607230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b1b9ed03c54d54b72ab94b747bc363'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.617822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b1b9ed03c54d54b72ab94b747bc363'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.724019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b1b9ed03c54d54b72ab94b747bc363'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.724846 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b1b9ed03c54d54b72ab94b747bc363'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:54.749007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1322/2679] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:54 2022] DELETE /identity/v3/users/78b1b9ed03c54d54b72ab94b747bc363 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:14:54.753292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:54.753542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:54.753927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:54.769464 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:54.776236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:54.777635 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:54.845573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1358/2680] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:54.856546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:54.858008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:54.925223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d55229-be3f-4aa0-948a-946afc36bc8c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:54.929858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:54.931672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:54.931672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:54.931672 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:54.940688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:54.942498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:54.952133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1323/2681] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:14:55.754138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:55.755974 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:55.768006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d8be0-53b1-42b9-9f42-e4e726a78c70 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:55.775288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:55.776020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:55.776409 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:55.776743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] PATH_INFO: `/v3/projects/5b84351a3a1d406ca68ed3c74d25ec6c` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:55.777895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b84351a3a1d406ca68ed3c74d25ec6c)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:55.779799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:55.946956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b84351a3a1d406ca68ed3c74d25ec6c', 'request_id': 'req-229e9606-d565-468c-a2a7-053d15b5e018'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:55.955357 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b84351a3a1d406ca68ed3c74d25ec6c', 'request_id': 'req-229e9606-d565-468c-a2a7-053d15b5e018'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:55.962232 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b84351a3a1d406ca68ed3c74d25ec6c', 'request_id': 'req-229e9606-d565-468c-a2a7-053d15b5e018'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.083460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] The token cache is being invalidate because project 5b84351a3a1d406ca68ed3c74d25ec6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:56.083754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.084396 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.085916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1359/2682] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:55 2022] DELETE /identity/v3/projects/5b84351a3a1d406ca68ed3c74d25ec6c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:56.089888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:56.100710 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837c7dd-cbf5-44bc-b095-f12d8103ed07 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:56.151033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:56.158020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:56.158657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:56.158937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:56.159208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] PATH_INFO: `/v3/users/e39777d27ef74d0fbe6ac2702a490676` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:56.161054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39777d27ef74d0fbe6ac2702a490676)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:56.162973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:56.423204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39777d27ef74d0fbe6ac2702a490676'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.423881 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39777d27ef74d0fbe6ac2702a490676'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.431186 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39777d27ef74d0fbe6ac2702a490676'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.511819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39777d27ef74d0fbe6ac2702a490676'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.512482 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39777d27ef74d0fbe6ac2702a490676'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:56.534444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1324/2683] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:56 2022] DELETE /identity/v3/users/e39777d27ef74d0fbe6ac2702a490676 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:57.079334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:57.081230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:57.095715 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e9606-d565-468c-a2a7-053d15b5e018 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:57.154114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:57.154771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:57.155054 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:57.155352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] PATH_INFO: `/v3/projects/b55c267a7174495fa7b81d7b11a3d00b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:57.156534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b55c267a7174495fa7b81d7b11a3d00b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:57.158425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:57.277736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:57.289565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b55c267a7174495fa7b81d7b11a3d00b', 'request_id': 'req-3a5aea0f-eea9-4bbd-9be7-537105ec2251'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.294854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b55c267a7174495fa7b81d7b11a3d00b', 'request_id': 'req-3a5aea0f-eea9-4bbd-9be7-537105ec2251'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.301152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b55c267a7174495fa7b81d7b11a3d00b', 'request_id': 'req-3a5aea0f-eea9-4bbd-9be7-537105ec2251'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.403738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] The token cache is being invalidate because project b55c267a7174495fa7b81d7b11a3d00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:57.404079 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.404863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.406177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1360/2684] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:57 2022] DELETE /identity/v3/projects/b55c267a7174495fa7b81d7b11a3d00b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:57.410133 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:57.411553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:57.425457 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c850b9d-8693-4f16-bcb0-5ed7143eae04 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:57.483059 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:57.483692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:57.483973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:57.484284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] PATH_INFO: `/v3/users/616d2de6b5324f57aea5c6faadbac327` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:57.485410 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616d2de6b5324f57aea5c6faadbac327)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:57.487342 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:57.761206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616d2de6b5324f57aea5c6faadbac327'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.761867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616d2de6b5324f57aea5c6faadbac327'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.768840 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616d2de6b5324f57aea5c6faadbac327'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.923421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616d2de6b5324f57aea5c6faadbac327'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.924176 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-12cdcff2-d25b-4329-93dc-f6de0a03c8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616d2de6b5324f57aea5c6faadbac327'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:57.946605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:57.955338 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1325/2685] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:14:57 2022] DELETE /identity/v3/users/616d2de6b5324f57aea5c6faadbac327 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:58.485297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:58.489647 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:58.509486 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aea0f-eea9-4bbd-9be7-537105ec2251 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:58.577993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:58.578764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:58.579074 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:58.579354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] PATH_INFO: `/v3/projects/8944029deb024cb3839a55878a10ffbf` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:58.580754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8944029deb024cb3839a55878a10ffbf)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:58.582692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:58.809994 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8944029deb024cb3839a55878a10ffbf', 'request_id': 'req-8cea29e3-cc86-4a72-9b65-20a087555cc8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:58.815397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8944029deb024cb3839a55878a10ffbf', 'request_id': 'req-8cea29e3-cc86-4a72-9b65-20a087555cc8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:58.821495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8944029deb024cb3839a55878a10ffbf', 'request_id': 'req-8cea29e3-cc86-4a72-9b65-20a087555cc8'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:58.951438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] The token cache is being invalidate because project 8944029deb024cb3839a55878a10ffbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:14:58.955320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:58.955320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8cea29e3-cc86-4a72-9b65-20a087555cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:14:58.955320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1361/2686] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:14:58 2022] DELETE /identity/v3/projects/8944029deb024cb3839a55878a10ffbf => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:58.961536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:58.962001 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:58.962112 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:58.982731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:58.983934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:59.044740 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1326/2687] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:59.054623 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.054903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.055184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.080394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:59.080394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:59.087264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1362/2688] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:59.091487 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.101755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc1746-67c3-4f23-8d6e-267129f22354 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.106393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.107045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.107325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.107620 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.114428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:14:59.114428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.114428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.122013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1327/2689] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:14:59.126363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.136754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb54881-afb7-42ed-856d-b004418c206a None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.141421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.142012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.142292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.142562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.143180 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.145095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.296616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1363/2690] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:14:59 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:59.300899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.311533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae7ee0-c624-429d-a47e-0811312ddfd4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.369569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.370179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.370459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.370731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.371611 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.373659 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.461422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1328/2691] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:14:59 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:14:59.465710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.477109 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7071d-e6df-483a-a0da-c5ab7186d1d4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.481705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.482306 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.482586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.482859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.483500 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.485147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.495546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1364/2692] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:14:59.499936 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.510449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdeade-5e0c-4cb6-888e-006ce7440c0d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.514960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.515712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.515991 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.516306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.516940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.518533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece075ab-50b3-4b6c-a972-499100b1372b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.529460 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1329/2693] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:14:59.533650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.544495 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881d47-a62c-4380-8b7e-ba30d4ff8200 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.549356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.549954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.550243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.550520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] PATH_INFO: `/v3/projects/3c7cedc9675943c2a164f6933c0ef14b/users/002aef0713a14c2ab580eee3d4cb6fc1/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.560184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:59.566565 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7cedc9675943c2a164f6933c0ef14b, user_id=002aef0713a14c2ab580eee3d4cb6fc1, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.569296 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.673327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1365/2694] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:14:59 2022] PUT /identity/v3/projects/3c7cedc9675943c2a164f6933c0ef14b/users/002aef0713a14c2ab580eee3d4cb6fc1/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:14:59.678312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477694d-2b5d-4ab4-b7a1-9e9cdf996237 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.678601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477694d-2b5d-4ab4-b7a1-9e9cdf996237 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.678883 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477694d-2b5d-4ab4-b7a1-9e9cdf996237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.700200 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-6477694d-2b5d-4ab4-b7a1-9e9cdf996237 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:14:59.701435 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-6477694d-2b5d-4ab4-b7a1-9e9cdf996237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:59.752027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:14:59.761179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1330/2695] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:14:59.770935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:14:59.773466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:14:59.828014 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ce6a-6ccd-47a3-b5a2-41e0e4e270ff admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:14:59.831836 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:14:59.832472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:14:59.832751 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:14:59.833020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:14:59.844930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:14:59.846404 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:14:59.856617 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1366/2696] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:01.486437 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54703-a4d4-4dd8-b2fb-e65ed0b0d7f3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.486990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54703-a4d4-4dd8-b2fb-e65ed0b0d7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.487312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54703-a4d4-4dd8-b2fb-e65ed0b0d7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:01.528490 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2ad54703-a4d4-4dd8-b2fb-e65ed0b0d7f3 None None] MFA Rules not processed for user `002aef0713a14c2ab580eee3d4cb6fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:01.530288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2ad54703-a4d4-4dd8-b2fb-e65ed0b0d7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:01.594424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1331/2697] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:01.599143 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:01.604915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.605198 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.605477 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:01.609882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d9cb7-2086-4b5a-903d-c944302375b8 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:01.614675 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:01.615297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.616137 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.616434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:01.617063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:01.619233 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:01.626091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] MFA Rules not processed for user `b67df0ce7f634db48cbd63d2bb0e5631`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:01.627295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:01.682495 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1332/2698] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:15:01.751713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1367/2699] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:15:01 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:01.753103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:01.764172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db88c6-c4c1-4876-acf6-8f8254597201 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:01.825805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:01.826418 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.826692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.826962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:01.827628 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:01.829821 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:01.940731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1333/2700] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:15:01 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:01.945437 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:01.955844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82927328-623f-4435-bd0c-02e89264e705 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:01.961590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:01.962189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.962466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.962736 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:01.963373 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:01.964993 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:01.976764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1368/2701] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:15:01.980859 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:01.991610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f479b3-4c5a-4b29-8d1c-f0d0c31577fc admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:01.996728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:01.997386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:01.997662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:01.997933 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] PATH_INFO: `/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3/users/62cd1795df2243268e6e7107b451508f/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:02.014150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802d4cd485e3416fa35ad4cbd6964dd3, user_id=62cd1795df2243268e6e7107b451508f, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:02.016983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:02.113546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:02.119250 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1334/2702] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:01 2022] PUT /identity/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3/users/62cd1795df2243268e6e7107b451508f/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:02.123414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:02.134116 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8078-6da4-4dd1-9338-2489614b66e0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:02.199520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:02.203583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:02.203583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:02.203583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:02.203583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:02.203583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2586abd-4bc8-4cef-afb0-90c4a9a71243 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:02.214473 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1369/2703] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:02.218927 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:02.229521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306685-c3e9-4043-82bd-837836ff92d0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:02.234433 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:02.235158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:02.235506 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:02.235847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] PATH_INFO: `/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3/users/62cd1795df2243268e6e7107b451508f/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:02.237858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802d4cd485e3416fa35ad4cbd6964dd3, user_id=62cd1795df2243268e6e7107b451508f, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:02.240723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:02.252045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1335/2704] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:02 2022] PUT /identity/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3/users/62cd1795df2243268e6e7107b451508f/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:03.778070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:03.778676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:03.778676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:03.806340 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] MFA Rules not processed for user `62cd1795df2243268e6e7107b451508f`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:03.807548 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:03.852599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:03.879511 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1370/2705] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:03.893077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:03.895662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:03.955590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd4180-85e3-4f58-acf3-b008e687ff77 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:03.959162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:03.959746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:03.960037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:03.960325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:04.017265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:04.018698 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:04.027990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1336/2706] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:06.010793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:06.010793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:06.022277 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af128121-afc3-4066-a11a-8b2b1881306d None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:06.026194 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:06.026810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:06.027242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:06.027520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:06.038460 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:06.040023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:06.050504 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1371/2707] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:15:14.023895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=53,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:16.049189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=16 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:23.014856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:23.083512 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bd675-d1a9-428d-9431-9e7247fe331b service cinder] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:23.088125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:23.088848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:23.089126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:23.089397 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] PATH_INFO: `/v3/users/002aef0713a14c2ab580eee3d4cb6fc1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:23.090502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002aef0713a14c2ab580eee3d4cb6fc1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:23.092478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:23.365358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002aef0713a14c2ab580eee3d4cb6fc1'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:23.366015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002aef0713a14c2ab580eee3d4cb6fc1'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:23.373371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002aef0713a14c2ab580eee3d4cb6fc1'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:23.487798 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002aef0713a14c2ab580eee3d4cb6fc1'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:23.488466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002aef0713a14c2ab580eee3d4cb6fc1'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:23.507258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1337/2708] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:15:23 2022] DELETE /identity/v3/users/002aef0713a14c2ab580eee3d4cb6fc1 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:23.850228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:23.864432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581af1c-d61b-4c06-8e8e-641fe93f1831 service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:23.919719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:23.920328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:23.920613 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:23.920891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] PATH_INFO: `/v3/projects/3c7cedc9675943c2a164f6933c0ef14b` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:23.922042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7cedc9675943c2a164f6933c0ef14b)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:23.923930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:24.204516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7cedc9675943c2a164f6933c0ef14b', 'request_id': 'req-34831b73-8a4c-4247-b620-7afe99509815'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.209986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7cedc9675943c2a164f6933c0ef14b', 'request_id': 'req-34831b73-8a4c-4247-b620-7afe99509815'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.215998 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7cedc9675943c2a164f6933c0ef14b', 'request_id': 'req-34831b73-8a4c-4247-b620-7afe99509815'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.339002 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] The token cache is being invalidate because project 3c7cedc9675943c2a164f6933c0ef14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:15:24.339292 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.340047 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.341573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1372/2709] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:15:23 2022] DELETE /identity/v3/projects/3c7cedc9675943c2a164f6933c0ef14b => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:24.345724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:24.347181 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:24.361395 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75057a56-a2b3-4fae-8ccc-4524eaf82746 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:24.419330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:24.419960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:24.420297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:24.420578 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] PATH_INFO: `/v3/users/62cd1795df2243268e6e7107b451508f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:24.421730 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62cd1795df2243268e6e7107b451508f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:24.423665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:24.445849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:24.678871 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62cd1795df2243268e6e7107b451508f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.679597 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62cd1795df2243268e6e7107b451508f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.686640 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62cd1795df2243268e6e7107b451508f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.859308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62cd1795df2243268e6e7107b451508f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.859913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-648708ba-21b2-43fc-9f07-e01a22f7aaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62cd1795df2243268e6e7107b451508f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:24.878220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1338/2710] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:15:24 2022] DELETE /identity/v3/users/62cd1795df2243268e6e7107b451508f => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:25.186632 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:25.188091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:25.203364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34831b73-8a4c-4247-b620-7afe99509815 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:25.265668 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:25.266297 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.266574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.266844 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] PATH_INFO: `/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.267947 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802d4cd485e3416fa35ad4cbd6964dd3)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:25.269886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:25.442697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:25.455171 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802d4cd485e3416fa35ad4cbd6964dd3', 'request_id': 'req-c4cd551c-3794-4166-bdaf-aea772648a30'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:25.460771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802d4cd485e3416fa35ad4cbd6964dd3', 'request_id': 'req-c4cd551c-3794-4166-bdaf-aea772648a30'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:25.467173 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802d4cd485e3416fa35ad4cbd6964dd3', 'request_id': 'req-c4cd551c-3794-4166-bdaf-aea772648a30'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:25.596671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] The token cache is being invalidate because project 802d4cd485e3416fa35ad4cbd6964dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:15:25.596999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:25.597568 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-c4cd551c-3794-4166-bdaf-aea772648a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:25.599169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1373/2711] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:15:25 2022] DELETE /identity/v3/projects/802d4cd485e3416fa35ad4cbd6964dd3 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:25.604156 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.604646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.604773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.624749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:25.626037 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:25.687121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1339/2712] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:25.695717 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.695942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.696254 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.715627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:25.716856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:25.724794 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1374/2713] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:25.729104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:25.738758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba56b6-2a22-4aef-a78a-3aed5ab64d06 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:25.744536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:25.745396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.745675 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.745947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.746593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:15:25.746899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:25.748529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:25.760918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1340/2714] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:15:25.772400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:25.781789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9cf44-c4b3-42fe-bb99-9b5a5683a89f None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:25.786972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:25.788023 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.788339 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.788616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.789248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:25.791119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:25.918399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1375/2715] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:25.922624 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:25.933661 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36f86c-ea50-4f28-8fc9-0dd6a5a725c2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:25.990273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:25.990652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:25.990934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:25.991214 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:25.991832 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:25.993927 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.140401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1341/2716] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:15:25 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:15:26.144546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.154929 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81964d9-5683-4948-8e52-2df907c78ebf admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.159375 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.159976 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.160313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.160593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.161255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.163010 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.173000 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1376/2717] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:26.176983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.187749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42a15-4746-45c4-ac85-e9f749dc118a admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.192547 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.194017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.194307 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.194589 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.195231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.196893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9422-c699-4975-ad13-e6003d655766 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.207577 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1342/2718] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:26.211873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.221671 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5025fd-35d5-488c-a61d-25278b75258f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.226487 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.227136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.227415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.227691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] PATH_INFO: `/v3/projects/0e50c204baab44a7aae470652ac1fad6/users/cd1410a4715f4296a4676de5c45f785e/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.245303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e50c204baab44a7aae470652ac1fad6, user_id=cd1410a4715f4296a4676de5c45f785e, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.248215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.391750 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1377/2719] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:26 2022] PUT /identity/v3/projects/0e50c204baab44a7aae470652ac1fad6/users/cd1410a4715f4296a4676de5c45f785e/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:26.402692 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.403025 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.403305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.411473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:26.431896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] MFA Rules not processed for user `cd1410a4715f4296a4676de5c45f785e`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:26.441873 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:26.499204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1343/2720] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:26.512793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.578471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf997e85-db97-4419-af28-ab7175c0f77e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.583153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.583864 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.584264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.584563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.585235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.587108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.724603 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1378/2721] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:15:26 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:26.728928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.739572 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb1d08-6da3-468c-83e1-fbe100b30a87 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.795449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.795974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.796284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.796559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.797175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.799098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.957605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1344/2722] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:15:26 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:15:26.962162 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:26.972095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350649dc-0caa-4e87-84dd-ae865b73531a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:26.977931 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:26.978557 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:26.978837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:26.979107 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:26.979746 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:26.981354 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:26.992255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1379/2723] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:26.996600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:27.006755 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55a407-5b7d-4522-b4dd-68494c00b8b1 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:27.011326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:27.011965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:27.012279 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:27.012552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] PATH_INFO: `/v3/projects/4c90afb5c19d4f4299630ef0d02d4103/users/0fbb763c28364c558e2da1e898de8f5d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:27.027657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c90afb5c19d4f4299630ef0d02d4103, user_id=0fbb763c28364c558e2da1e898de8f5d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:27.030313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:27.134948 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1345/2724] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:26 2022] PUT /identity/v3/projects/4c90afb5c19d4f4299630ef0d02d4103/users/0fbb763c28364c558e2da1e898de8f5d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:15:27.139661 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:27.150541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eb97-d840-42bb-9560-472096d008ac admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:27.178816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:27.205898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:27.206523 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:27.206805 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:27.207094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:27.207735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:27.210063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a113a0-ba7c-47ef-b989-7f83e69aa286 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:27.220013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1380/2725] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:15:27.224937 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:27.235361 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595359a-16bb-4d4f-8958-864763108621 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:27.240074 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:27.240772 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:27.241055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:27.241357 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] PATH_INFO: `/v3/projects/4c90afb5c19d4f4299630ef0d02d4103/users/0fbb763c28364c558e2da1e898de8f5d/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:27.243263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c90afb5c19d4f4299630ef0d02d4103, user_id=0fbb763c28364c558e2da1e898de8f5d, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:27.246299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:27.257284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1346/2726] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:15:27 2022] PUT /identity/v3/projects/4c90afb5c19d4f4299630ef0d02d4103/users/0fbb763c28364c558e2da1e898de8f5d/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:27.324960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:27.325242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:27.325526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:27.353422 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] MFA Rules not processed for user `0fbb763c28364c558e2da1e898de8f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:27.354536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:27.427783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1381/2727] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:27.438594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:27.440350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:27.493609 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:27.499430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f5c4-b7f5-4412-a006-a4dbb3a94447 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:27.502950 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:27.506054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:27.506054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:27.506054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:27.559626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:27.561175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2878ca-1ed6-4330-98a9-df919a1bffb0 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:27.570469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1347/2728] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:30.834319 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:30.897062 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f77ac-c11a-49ab-a9c2-b37ab36ae258 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:30.903573 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:30.904235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:30.904517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:30.904790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] PATH_INFO: `/v3/users/cd1410a4715f4296a4676de5c45f785e` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:30.905906 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1410a4715f4296a4676de5c45f785e)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:30.907800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:31.107442 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1410a4715f4296a4676de5c45f785e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:31.108087 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1410a4715f4296a4676de5c45f785e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:31.113225 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:31.115678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1410a4715f4296a4676de5c45f785e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:31.185541 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1410a4715f4296a4676de5c45f785e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:31.186157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1410a4715f4296a4676de5c45f785e'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:31.203696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1382/2729] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:15:30 2022] DELETE /identity/v3/users/cd1410a4715f4296a4676de5c45f785e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:31.209048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:31.209555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:31.209555 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:31.228408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:15:31.229598 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:31.288119 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1348/2730] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:15:31.298105 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:31.299552 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:31.357182 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03cffc-80de-4f6a-9c01-f903814d8c6e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:31.360790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:31.361434 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:31.361737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:31.362059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:31.372028 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:31.373469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:31.383686 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1383/2731] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:15:31.832761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:31.834204 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:31.846932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a9fb3-cd3c-4f15-8f21-4f9ae2e266d7 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:31.853860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:31.854508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:31.854785 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:31.855056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] PATH_INFO: `/v3/projects/0e50c204baab44a7aae470652ac1fad6` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:31.856236 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e50c204baab44a7aae470652ac1fad6)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:31.859033 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:32.009866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e50c204baab44a7aae470652ac1fad6', 'request_id': 'req-fcab68ba-9fe4-43a0-a633-2b5343b3097d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.015390 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e50c204baab44a7aae470652ac1fad6', 'request_id': 'req-fcab68ba-9fe4-43a0-a633-2b5343b3097d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.145448 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e50c204baab44a7aae470652ac1fad6', 'request_id': 'req-fcab68ba-9fe4-43a0-a633-2b5343b3097d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.158098 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] The token cache is being invalidate because project 0e50c204baab44a7aae470652ac1fad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:15:32.164925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.164925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.164925 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1349/2732] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:15:31 2022] DELETE /identity/v3/projects/0e50c204baab44a7aae470652ac1fad6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:32.165485 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:32.175275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f763fd-c58c-4b57-9861-5da1c5212758 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:32.231204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:32.231828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:32.232108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:32.232414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] PATH_INFO: `/v3/users/0fbb763c28364c558e2da1e898de8f5d` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:32.233516 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbb763c28364c558e2da1e898de8f5d)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:32.235416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:32.489955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbb763c28364c558e2da1e898de8f5d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.490630 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbb763c28364c558e2da1e898de8f5d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.499985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbb763c28364c558e2da1e898de8f5d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.585320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbb763c28364c558e2da1e898de8f5d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.585933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-f0f50379-8017-47cf-b09c-51549369a6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbb763c28364c558e2da1e898de8f5d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:32.596599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:32.603946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1384/2733] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:15:32 2022] DELETE /identity/v3/users/0fbb763c28364c558e2da1e898de8f5d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:33.157411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:15:33.158867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:15:33.172425 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab68ba-9fe4-43a0-a633-2b5343b3097d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:15:33.195016 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:33.230284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:15:33.230915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:15:33.231194 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:15:33.231469 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] PATH_INFO: `/v3/projects/4c90afb5c19d4f4299630ef0d02d4103` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:15:33.232638 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c90afb5c19d4f4299630ef0d02d4103)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:15:33.234553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:15:33.385926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c90afb5c19d4f4299630ef0d02d4103', 'request_id': 'req-26b08c7c-6c4a-41a8-b986-05ed600a334e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:33.391408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c90afb5c19d4f4299630ef0d02d4103', 'request_id': 'req-26b08c7c-6c4a-41a8-b986-05ed600a334e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:33.511172 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c90afb5c19d4f4299630ef0d02d4103', 'request_id': 'req-26b08c7c-6c4a-41a8-b986-05ed600a334e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:33.522709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] The token cache is being invalidate because project 4c90afb5c19d4f4299630ef0d02d4103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:15:33.522990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:33.523608 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:15:33.525115 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1350/2734] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:15:33 2022] DELETE /identity/v3/projects/4c90afb5c19d4f4299630ef0d02d4103 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:15:42.956583 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:15:43.683139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:16:06.838067 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:06.842881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:06.842881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:06.858316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] MFA Rules not processed for user `b67df0ce7f634db48cbd63d2bb0e5631`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:16:06.859505 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:06.915095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1385/2735] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:16:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:16:06.952848 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:16:06.954378 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:07.010421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08c7c-6c4a-41a8-b986-05ed600a334e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:16:07.013979 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:16:07.014580 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:07.014861 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:07.015147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:07.016855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:07.087139 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:16:07.088655 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:16:07.099824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1351/2736] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:16:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:16:07.828745 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:16:07.830248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:07.889403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a12c4-1226-4510-912d-caeddca4b9a9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:16:07.893058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:16:07.893753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:16:07.893936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:16:07.894249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:16:07.895946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:16:07.909901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:16:07.911329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:16:07.924914 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1386/2737] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:16:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:16:17.094637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=54 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:16:17.923494 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=51 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:08.069728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:08.071245 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:08.085573 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef2bd-45e4-4eb6-84e6-af78097f83de service glance] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:08.091312 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:08.092045 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:08.092483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:08.092643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:08.094389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:08.107499 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:08.109077 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:08.141646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1352/2738] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:17.923971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:17.926149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:17.994255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d2d0-a193-4457-8297-557b6e975c58 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:17.998660 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:17.999284 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:17.999610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:17.999888 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] PATH_INFO: `/v3/users/b67df0ce7f634db48cbd63d2bb0e5631` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:18.001042 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67df0ce7f634db48cbd63d2bb0e5631)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:18.002928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:18.121717 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:18.254376 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67df0ce7f634db48cbd63d2bb0e5631'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.255025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67df0ce7f634db48cbd63d2bb0e5631'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.262020 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67df0ce7f634db48cbd63d2bb0e5631'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.360767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67df0ce7f634db48cbd63d2bb0e5631'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.361390 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-3669877e-af11-48d7-b1e7-36412361db1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67df0ce7f634db48cbd63d2bb0e5631'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.382808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1387/2739] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:17 2022] DELETE /identity/v3/users/b67df0ce7f634db48cbd63d2bb0e5631 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:18.674242 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:18.685324 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662d575-447d-49c0-8f7b-a6caf38636ba service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:18.745701 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:18.746347 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:18.746806 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:18.747081 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] PATH_INFO: `/v3/projects/7d495b3e011c49388cc8cff98aeb1d73` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:18.748314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d495b3e011c49388cc8cff98aeb1d73)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:18.750213 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:18.968044 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d495b3e011c49388cc8cff98aeb1d73', 'request_id': 'req-f0672bbe-2940-4766-8e8d-478cc5f2874c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:18.973965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d495b3e011c49388cc8cff98aeb1d73', 'request_id': 'req-f0672bbe-2940-4766-8e8d-478cc5f2874c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:19.074632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d495b3e011c49388cc8cff98aeb1d73', 'request_id': 'req-f0672bbe-2940-4766-8e8d-478cc5f2874c'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:19.086665 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] The token cache is being invalidate because project 7d495b3e011c49388cc8cff98aeb1d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:19.086974 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:19.087603 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-f0672bbe-2940-4766-8e8d-478cc5f2874c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:19.089183 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1353/2740] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:18 2022] DELETE /identity/v3/projects/7d495b3e011c49388cc8cff98aeb1d73 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:19.094045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.094515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.094642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.113793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:19.114967 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:19.175551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1388/2741] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:19.184275 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.184732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.184957 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.205088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:19.206314 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:19.215466 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1354/2742] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:17:19.219862 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.230440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e01cea-8be2-49e5-8b3b-b6ae804a28c6 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.235018 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.235612 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.235884 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.236166 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.236820 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:17:19.237118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.238935 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.250591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1389/2743] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:17:19.255532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.264833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4419b-b75e-4a93-8e8b-f954e242bc76 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.270150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.270864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.271141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.271409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.272039 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.273965 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.379546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1355/2744] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:19.383811 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.393852 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960adeba-e876-4324-8f1a-1c337139f226 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.397581 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:19.450449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.450992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.451270 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.451544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.452196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.454104 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.558728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1390/2745] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:19.562942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.573792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e653-3ef3-4133-846e-c1d9a8e99cc9 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.617491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.617491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.617491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.617491 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.618040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.618040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.618040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1356/2746] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:19.620196 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.630329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95502835-7358-4d1d-a4f7-f0b558df3d07 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.634958 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.635547 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.635825 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.636096 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.636792 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.638355 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db264c21-a905-4c4a-82f5-2bd57eb28f7e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.650094 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1391/2747] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:19.654162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.664934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdf69f-6603-42a7-9315-62e40ca5e25c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.669669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.670315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.670595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.670896 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] PATH_INFO: `/v3/projects/5a532c21696841388651b3381eeacbc9/users/eddeca1b79884db19ca296afb88a9376/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.685752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a532c21696841388651b3381eeacbc9, user_id=eddeca1b79884db19ca296afb88a9376, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.688445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:19.778334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1357/2748] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:19 2022] PUT /identity/v3/projects/5a532c21696841388651b3381eeacbc9/users/eddeca1b79884db19ca296afb88a9376/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:19.789295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.789615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.789894 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.819383 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] MFA Rules not processed for user `eddeca1b79884db19ca296afb88a9376`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:19.820627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:19.883653 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1392/2749] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:19.888304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:19.907863 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:19.950949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f673a4-3a9d-4b21-9c86-e91898b646b4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:19.955686 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:19.956319 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:19.956604 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:19.956895 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:19.958175 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:19.960084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.080095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1358/2750] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:19 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:20.083969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.094674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e27079-e4de-45f2-b10d-098c2e500ade None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.151558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.152184 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.152480 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.152756 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.153374 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.155271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.272429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1393/2751] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:20 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:20.276571 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.287142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8baa9-f6bb-4762-b674-a0f96a611cfb admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.291816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.292456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.292738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.293012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.293688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.295326 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.306720 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1359/2752] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:20.310778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.317748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:20.321424 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb38652-9125-48a9-be97-289a12befbde admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.326130 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.333136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.333136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.333136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] PATH_INFO: `/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26/users/f0c5d46ea55a46b3b4ed34bf78247f28/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.343295 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3a23989f2b44b3beb2d4730eb2ab26, user_id=f0c5d46ea55a46b3b4ed34bf78247f28, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.346082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.410036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1394/2753] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:20 2022] PUT /identity/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26/users/f0c5d46ea55a46b3b4ed34bf78247f28/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:20.414722 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.425031 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51bf93-f4c0-4f75-b221-c1692a34ef86 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.485130 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.485744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.486027 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.486322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.487109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.489658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8909f-b6da-4b23-a5f9-970f3333904d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.499943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1360/2754] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:20.504352 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.514461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbce54-b7e4-43db-a8f8-c662b0de1062 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.518956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.519711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.519992 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.520303 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] PATH_INFO: `/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26/users/f0c5d46ea55a46b3b4ed34bf78247f28/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.522200 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3a23989f2b44b3beb2d4730eb2ab26, user_id=f0c5d46ea55a46b3b4ed34bf78247f28, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.524868 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.534915 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1395/2755] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:20 2022] PUT /identity/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26/users/f0c5d46ea55a46b3b4ed34bf78247f28/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:20.545759 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.546051 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.546331 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.574461 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] MFA Rules not processed for user `f0c5d46ea55a46b3b4ed34bf78247f28`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:20.575681 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:20.643721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1361/2756] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:20.655006 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.656768 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:20.714673 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93ca0-f33f-45f1-ab48-f659bdb18d84 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.718293 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.718886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.719164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.719438 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.728786 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.730193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.740098 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1396/2757] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:20.907914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:20.909408 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:20.922392 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807967-f6c7-4132-9415-14fdb519f8e3 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:20.926134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:20.926752 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:20.927026 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:20.927300 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:20.937421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:20.985833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:20.987280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f127d-5f6b-4ae9-84cd-2eebb525f250 service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:20.996430 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1362/2758] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:26.025038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:26.093414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57041f8-abd6-49c1-98e3-0c61def17ffb service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:26.098515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:26.099521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:26.100029 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:26.100546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] PATH_INFO: `/v3/users/eddeca1b79884db19ca296afb88a9376` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:26.102540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddeca1b79884db19ca296afb88a9376)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:26.104529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:26.299785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddeca1b79884db19ca296afb88a9376'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:26.300439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddeca1b79884db19ca296afb88a9376'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:26.308051 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddeca1b79884db19ca296afb88a9376'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:26.374692 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddeca1b79884db19ca296afb88a9376'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:26.375403 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddeca1b79884db19ca296afb88a9376'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:26.383826 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:26.394529 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1397/2759] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:26 2022] DELETE /identity/v3/users/eddeca1b79884db19ca296afb88a9376 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:26.399104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:26.405560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:26.405560 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:26.418644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:26.419886 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:26.478174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1363/2760] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:26.487827 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:26.489294 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:26.545542 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12675872-5d20-4203-abd2-f4f909e15526 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:26.549118 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:26.549709 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:26.550038 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:26.550253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:26.559628 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:26.561069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:26.570419 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1398/2761] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:27.008728 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:27.010154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:27.022660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7184c-8d1e-4cc5-bae4-24b1663bbd95 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:27.027315 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:27.028093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:27.028403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:27.028677 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] PATH_INFO: `/v3/projects/5a532c21696841388651b3381eeacbc9` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:27.029794 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a532c21696841388651b3381eeacbc9)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:27.031746 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:27.133411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a532c21696841388651b3381eeacbc9', 'request_id': 'req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.142421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a532c21696841388651b3381eeacbc9', 'request_id': 'req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.228876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a532c21696841388651b3381eeacbc9', 'request_id': 'req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.241079 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] The token cache is being invalidate because project 5a532c21696841388651b3381eeacbc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:27.241369 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.242276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.243805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1364/2762] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:27 2022] DELETE /identity/v3/projects/5a532c21696841388651b3381eeacbc9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:27.247810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:27.257537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da596a7-958a-40d2-a297-ed90bffa420c service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:27.313221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:27.313841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:27.314115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:27.314385 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] PATH_INFO: `/v3/users/f0c5d46ea55a46b3b4ed34bf78247f28` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:27.315479 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c5d46ea55a46b3b4ed34bf78247f28)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:27.317467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:27.509761 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c5d46ea55a46b3b4ed34bf78247f28'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.510415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c5d46ea55a46b3b4ed34bf78247f28'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.517507 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c5d46ea55a46b3b4ed34bf78247f28'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.589189 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c5d46ea55a46b3b4ed34bf78247f28'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.589814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bc448af2-008c-41c6-9f8c-d9e4361293b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c5d46ea55a46b3b4ed34bf78247f28'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:27.607472 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1399/2763] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:27 2022] DELETE /identity/v3/users/f0c5d46ea55a46b3b4ed34bf78247f28 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:28.042199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.043773 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:28.058092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef92b8-63a2-4adf-ab15-0f1a4824797e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.092447 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:28.117689 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.118330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.118641 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.118922 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] PATH_INFO: `/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.120080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3a23989f2b44b3beb2d4730eb2ab26)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.122018 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.269013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3a23989f2b44b3beb2d4730eb2ab26', 'request_id': 'req-95dc60fd-718f-4776-8035-111604a402ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:28.274926 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3a23989f2b44b3beb2d4730eb2ab26', 'request_id': 'req-95dc60fd-718f-4776-8035-111604a402ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:28.372063 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3a23989f2b44b3beb2d4730eb2ab26', 'request_id': 'req-95dc60fd-718f-4776-8035-111604a402ff'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:28.383297 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] The token cache is being invalidate because project 5e3a23989f2b44b3beb2d4730eb2ab26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:28.383584 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:28.384266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-95dc60fd-718f-4776-8035-111604a402ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:28.385774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1365/2764] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:28 2022] DELETE /identity/v3/projects/5e3a23989f2b44b3beb2d4730eb2ab26 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:28.390703 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.391192 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.391308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.410670 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:28.412654 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:28.417828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:28.471135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1400/2765] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:28.479735 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.480021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.480335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.499838 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:28.501066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:28.509167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1366/2766] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:28.513416 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.524537 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc9fc-587f-477e-81eb-f84ec44d6e99 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.529100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.529693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.529971 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.530242 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.530887 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:17:28.531253 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.532881 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.544886 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1401/2767] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:17:28.549042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.558142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0b54b-bd0d-4c89-8541-10ebc867e0fe None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.562673 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.563256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.563529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.563797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.564445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.566334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.734600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1367/2768] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:28 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:28.738311 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.749217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880c861-894f-4560-bfec-a01ee2c53d8a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.804880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.805476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.805752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.806021 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.806640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.808575 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.912710 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1402/2769] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:28 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:28.916875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.927943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3daf6e-cad6-4824-bb66-3f29b5fe3c10 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.937285 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.937910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.938192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.938467 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.939117 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.940758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.951174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1368/2770] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:28.955266 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.965207 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6c5c-cdab-4794-b6a4-6e6e2b35641a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:28.969713 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:28.970299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:28.970577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:28.970849 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:28.971483 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:28.973091 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56f76-dded-45e1-9efc-7928b73d1f5f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:28.983948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1403/2771] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:28.988008 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:28.998652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be055a4-d094-4ce3-9bfc-0946eef56c3e admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.003167 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.003805 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.004083 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.004386 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] PATH_INFO: `/v3/projects/b12b41b924db423eac64842a79618e51/users/57151a0ed0df4f2caf3d10ad23018c43/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.020120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12b41b924db423eac64842a79618e51, user_id=57151a0ed0df4f2caf3d10ad23018c43, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.023463 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.094860 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1369/2772] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:28 2022] PUT /identity/v3/projects/b12b41b924db423eac64842a79618e51/users/57151a0ed0df4f2caf3d10ad23018c43/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:29.105726 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.106012 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.106280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.133595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] MFA Rules not processed for user `57151a0ed0df4f2caf3d10ad23018c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:29.134790 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:29.200429 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1404/2773] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:29.205075 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.266767 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d9722-066d-402d-811e-4604262447dd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.272360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.272975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.273257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.273532 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.274152 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.276055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.281280 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:29.370600 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1370/2774] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:29 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:29.374696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.386075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3245ae0-51cc-4dbf-96dc-6e68a22ea732 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.390314 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:29.442102 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.443353 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.443624 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.443889 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.444593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.446532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.532800 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1405/2775] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:29 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:29.536971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.547333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e422a3-3e6f-468a-8bd3-6e6f9a6fbf2d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.552224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.552910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.553185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.553459 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.554134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.555856 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.565882 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1371/2776] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:29.575153 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.585534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196b4cd-cbb2-4dc6-b4b3-8f278d84acb3 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.590095 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.590679 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.590955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.591255 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.591890 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.593526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732705d1-30df-477e-a804-bb91d8d7abf6 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.603695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1406/2777] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:29.607799 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.618451 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099df5b-17b9-4ad2-a2cb-d5afcbeb2703 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.623017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.623653 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.623940 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.624308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] PATH_INFO: `/v3/projects/2554bf4b288440fea92c1b9c91df434f/users/876251fee0db4e769d4183104e9637ab/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.639989 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2554bf4b288440fea92c1b9c91df434f, user_id=876251fee0db4e769d4183104e9637ab, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.642847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:29.723456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1372/2778] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:29 2022] PUT /identity/v3/projects/2554bf4b288440fea92c1b9c91df434f/users/876251fee0db4e769d4183104e9637ab/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:29.789007 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.789224 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.789455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.818561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] MFA Rules not processed for user `876251fee0db4e769d4183104e9637ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:29.819799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:29.883461 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1407/2779] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:29.887969 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:29.959055 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fd1bb-03c5-47f0-a6ea-b8b79aa91f6d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:29.963797 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:29.964434 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:29.964714 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:29.964986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:29.965610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:29.967522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.062276 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1373/2780] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:29 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:30.070515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.081211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6ff7c-da74-44a0-9253-8a611d9ae566 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.139513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.140119 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.140448 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.140743 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.141358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.143269 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.229754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1408/2781] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:30 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:30.233782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.244449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decba1-572e-413e-95e9-d052429e7e35 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.249019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.255304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.263308 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1374/2782] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:30.267367 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.274397 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:30.278187 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c555-2c66-40f4-9008-67374ee7e8a4 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.282788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.283933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.283933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.283933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] PATH_INFO: `/v3/projects/b8d29f65c58f42f3bea2603e251b77a0/users/1ca53ce7571240b0b867ec4598d998f3/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.298359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d29f65c58f42f3bea2603e251b77a0, user_id=1ca53ce7571240b0b867ec4598d998f3, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.301059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.384131 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1409/2783] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:30 2022] PUT /identity/v3/projects/b8d29f65c58f42f3bea2603e251b77a0/users/1ca53ce7571240b0b867ec4598d998f3/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:30.388177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.398923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01858de-cc62-4de3-97fc-84abb307458d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.453884 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.454488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.454771 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.455047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.455688 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.457398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dac986-4c4c-41be-ae2b-347067644474 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.467602 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1375/2784] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:30.471708 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.481937 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295265f9-32a8-492b-9421-af1f6e84d100 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.486440 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.487076 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.487348 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.487615 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] PATH_INFO: `/v3/projects/b8d29f65c58f42f3bea2603e251b77a0/users/1ca53ce7571240b0b867ec4598d998f3/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.489688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d29f65c58f42f3bea2603e251b77a0, user_id=1ca53ce7571240b0b867ec4598d998f3, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.492561 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.502470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1410/2785] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:30 2022] PUT /identity/v3/projects/b8d29f65c58f42f3bea2603e251b77a0/users/1ca53ce7571240b0b867ec4598d998f3/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:30.513099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.513384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.513658 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.541913 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] MFA Rules not processed for user `1ca53ce7571240b0b867ec4598d998f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:30.543163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:30.553038 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:30.610754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1376/2786] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:30.622788 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.624459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:30.682423 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deedd14-d4a2-4ea9-9b2d-8de07d7c598e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.685956 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.686559 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.686863 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.687135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.696739 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.698149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.707265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1411/2787] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:30.803449 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:30.804932 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:30.817258 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a018-1c86-4739-9a1e-23ad8c8e270c None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:30.822745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:30.830121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:30.830121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:30.830121 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:30.880567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:30.882000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff654-023e-44d5-82a7-6adf716b1cfd service cinder] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:30.890744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1377/2788] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:17:34.980734 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:35.052902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47677b-ff57-4815-8f17-5107fc0d8ecb service cinder] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:35.057733 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:35.058389 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:35.058808 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:35.059113 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] PATH_INFO: `/v3/users/57151a0ed0df4f2caf3d10ad23018c43` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:35.060323 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57151a0ed0df4f2caf3d10ad23018c43)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:35.062265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:35.297943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57151a0ed0df4f2caf3d10ad23018c43'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:35.298650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57151a0ed0df4f2caf3d10ad23018c43'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:35.306362 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57151a0ed0df4f2caf3d10ad23018c43'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:35.351517 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57151a0ed0df4f2caf3d10ad23018c43'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:35.352135 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57151a0ed0df4f2caf3d10ad23018c43'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:35.359535 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:35.371386 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1412/2789] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:34 2022] DELETE /identity/v3/users/57151a0ed0df4f2caf3d10ad23018c43 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:35.375903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:35.376431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:35.376564 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:35.397100 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:35.398399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:35.458303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1378/2790] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:35.467965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:35.469642 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:35.527401 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343454d2-523d-4577-b665-55f2f9342ba1 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:35.530982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:35.531576 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:35.531853 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:35.532122 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:35.541543 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:35.542999 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:35.552984 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1413/2791] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:17:35.984644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:35.986164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:35.999528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768cda4-af32-4cdf-be29-7a92629b08aa None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:36.004124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:36.005503 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:36.005778 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:36.006054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] PATH_INFO: `/v3/projects/b12b41b924db423eac64842a79618e51` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:36.007295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b12b41b924db423eac64842a79618e51)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:36.009291 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:36.149396 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:36.155332 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12b41b924db423eac64842a79618e51', 'request_id': 'req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.161834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12b41b924db423eac64842a79618e51', 'request_id': 'req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.263380 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12b41b924db423eac64842a79618e51', 'request_id': 'req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.274648 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] The token cache is being invalidate because project b12b41b924db423eac64842a79618e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:36.275293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.275570 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.277086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1379/2792] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:35 2022] DELETE /identity/v3/projects/b12b41b924db423eac64842a79618e51 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:36.281048 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:36.291315 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c914b0-765b-40f2-b723-e9e49627fd16 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:36.347190 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:36.347809 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:36.348084 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:36.348927 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] PATH_INFO: `/v3/users/876251fee0db4e769d4183104e9637ab` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:36.350013 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876251fee0db4e769d4183104e9637ab)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:36.351909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:36.559211 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876251fee0db4e769d4183104e9637ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.559859 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876251fee0db4e769d4183104e9637ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.566744 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876251fee0db4e769d4183104e9637ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.628898 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876251fee0db4e769d4183104e9637ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.629610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876251fee0db4e769d4183104e9637ab'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:36.648058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1414/2793] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:36 2022] DELETE /identity/v3/users/876251fee0db4e769d4183104e9637ab => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:37.010850 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:37.012360 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:37.026533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef80e85-cb73-4941-b6ec-c3abd8ec1e2d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:37.085346 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:37.085976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:37.086254 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:37.086558 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] PATH_INFO: `/v3/projects/2554bf4b288440fea92c1b9c91df434f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:37.087703 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2554bf4b288440fea92c1b9c91df434f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:37.089693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:37.198124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2554bf4b288440fea92c1b9c91df434f', 'request_id': 'req-0a3c3e92-4d57-4b01-b39d-21a17522839d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.203637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2554bf4b288440fea92c1b9c91df434f', 'request_id': 'req-0a3c3e92-4d57-4b01-b39d-21a17522839d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.277203 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2554bf4b288440fea92c1b9c91df434f', 'request_id': 'req-0a3c3e92-4d57-4b01-b39d-21a17522839d'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.289474 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] The token cache is being invalidate because project 2554bf4b288440fea92c1b9c91df434f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:37.293888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.293888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.293888 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1380/2794] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:37 2022] DELETE /identity/v3/projects/2554bf4b288440fea92c1b9c91df434f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:17:37.295877 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:37.297356 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:37.311133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa994e5-a8cd-4b8c-935d-d45f0b3cef32 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:37.319807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:37.370025 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:37.370664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:37.370938 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:37.371203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] PATH_INFO: `/v3/users/1ca53ce7571240b0b867ec4598d998f3` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:37.372343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca53ce7571240b0b867ec4598d998f3)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:37.374272 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:37.531175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca53ce7571240b0b867ec4598d998f3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.531829 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca53ce7571240b0b867ec4598d998f3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.538985 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca53ce7571240b0b867ec4598d998f3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.602618 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca53ce7571240b0b867ec4598d998f3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.603251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-bd1a6a2b-949c-4542-84b3-cd57c3c2c11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca53ce7571240b0b867ec4598d998f3'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:37.620468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1415/2795] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:37 2022] DELETE /identity/v3/users/1ca53ce7571240b0b867ec4598d998f3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:38.038867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.041852 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:38.054727 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c3e92-4d57-4b01-b39d-21a17522839d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.112880 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.113522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.113802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.114076 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] PATH_INFO: `/v3/projects/b8d29f65c58f42f3bea2603e251b77a0` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.115815 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d29f65c58f42f3bea2603e251b77a0)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.118561 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.120070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:38.223862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d29f65c58f42f3bea2603e251b77a0', 'request_id': 'req-8367e048-baee-4aa7-b1eb-e73a4f56189f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:38.229417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d29f65c58f42f3bea2603e251b77a0', 'request_id': 'req-8367e048-baee-4aa7-b1eb-e73a4f56189f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:38.308971 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d29f65c58f42f3bea2603e251b77a0', 'request_id': 'req-8367e048-baee-4aa7-b1eb-e73a4f56189f'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:38.321093 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] The token cache is being invalidate because project b8d29f65c58f42f3bea2603e251b77a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:38.321383 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:38.322012 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-8367e048-baee-4aa7-b1eb-e73a4f56189f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:38.323515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1381/2796] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:38 2022] DELETE /identity/v3/projects/b8d29f65c58f42f3bea2603e251b77a0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:38.329489 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.329982 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.330112 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.349345 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:38.350669 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:38.408738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1416/2797] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:38.417567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.417857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.418161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.438691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:38.439909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:38.448058 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1382/2798] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:38.452274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.462562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9291f-c9ab-4ecd-b421-3b180782a420 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.467044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.467637 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.467916 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.468217 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.468892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81785) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:17:38.469198 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.470933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.483288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1417/2799] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 25 03:17:38.487749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.497889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034aa8f-d405-4712-a1ef-2faaddc3c654 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.502671 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.503284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.503588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.503857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.504507 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.506528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.627724 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1383/2800] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:38 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:38.632358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.642147 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7048-26fc-4e05-a036-dfcca021cc6b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.654026 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:38.698165 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.698793 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.699069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.699337 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.699969 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.701966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.794801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1418/2801] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:38 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:38.798924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.809302 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04949f8-b1d7-4d82-a635-d8a098272d67 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.814129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.814716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.814992 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.815259 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.815891 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.817517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.827335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1384/2802] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:38.831157 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.841725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda98ef-bebc-44c2-bfb7-6633d26a5e41 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.848477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.848477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.848477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.848477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.848872 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.850527 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5513f1-8011-4d85-9a5e-aad487dbdc73 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.861069 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1419/2803] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:17:38.865230 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:38.875309 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d2b1-0325-45c2-b054-9eaf53df84a4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:38.879889 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:38.880522 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.880816 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:38.881220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] PATH_INFO: `/v3/projects/e6852f7c61a54191b8ee0c3c9d97cb51/users/06000abb4d714d66a05dce1b3edbe2eb/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:38.895104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6852f7c61a54191b8ee0c3c9d97cb51, user_id=06000abb4d714d66a05dce1b3edbe2eb, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:38.898790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:38.986261 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1385/2804] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:38 2022] PUT /identity/v3/projects/e6852f7c61a54191b8ee0c3c9d97cb51/users/06000abb4d714d66a05dce1b3edbe2eb/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:38.999553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:38.999840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.004738 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.027219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] MFA Rules not processed for user `06000abb4d714d66a05dce1b3edbe2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:39.028532 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:39.090755 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1420/2805] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:17:39.095350 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.157557 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104886-8902-4a68-a242-947f170be8b2 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.162304 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.162949 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.163231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.163504 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.164142 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.166085 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.249521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1386/2806] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:39 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:39.253674 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.265590 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c1485-c007-4457-9bbe-d031029ebd76 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.322066 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.322666 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.322948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.323250 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] PATH_INFO: `/v3/users` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.323873 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.325846 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.411819 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1421/2807] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:39 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:39.415899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.426217 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3000d9a-6922-41c5-ab06-07c5d342a761 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.431042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.431633 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.431914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.432206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.432869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.434508 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.439650 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:39.445857 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1387/2808] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:39.450077 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.460766 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e84f5-372a-4359-af60-0c0edc489793 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.465626 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.466299 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.466578 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.466847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] PATH_INFO: `/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef/users/d44aaf96280a4c138efb0b57a026ece0/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.479499 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:39.482539 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed44be57a674a72a37e29cd53f4b9ef, user_id=d44aaf96280a4c138efb0b57a026ece0, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.485317 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.606455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1422/2809] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:39 2022] PUT /identity/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef/users/d44aaf96280a4c138efb0b57a026ece0/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:39.610393 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.620983 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79abe6-f6d4-43a7-9202-5f9799005946 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.675330 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.676293 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.676574 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.676849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.678064 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.679729 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f8455-6913-4cac-950d-7cd45cdc7501 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.689534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1388/2810] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:39.694525 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.705085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c9a3f-35f6-49c8-9498-0040e0724042 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.709778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.710414 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.710688 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.710959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] PATH_INFO: `/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef/users/d44aaf96280a4c138efb0b57a026ece0/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.712885 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed44be57a674a72a37e29cd53f4b9ef, user_id=d44aaf96280a4c138efb0b57a026ece0, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.715456 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.725562 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1423/2811] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:39 2022] PUT /identity/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef/users/d44aaf96280a4c138efb0b57a026ece0/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:39.735542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.735833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.736114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.765228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] MFA Rules not processed for user `d44aaf96280a4c138efb0b57a026ece0`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:39.766428 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:39.834118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1389/2812] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:39.844986 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:39.846732 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:39.904705 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a53bd-a770-43f4-a7bd-0f34a4e15f83 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:39.908338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:39.908926 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:39.909201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:39.909468 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:39.920139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:39.921566 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fa447-fb51-475f-acb9-27a5b926915c service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:39.932400 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1424/2813] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:40.472238 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:40.534053 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802fa96-983e-4943-92c5-9abd31d79ccd None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:40.538644 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:40.539267 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:40.539546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:40.539819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] PATH_INFO: `/v3/users/06000abb4d714d66a05dce1b3edbe2eb` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:40.540961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06000abb4d714d66a05dce1b3edbe2eb)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:40.542875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:40.564893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:40.739515 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06000abb4d714d66a05dce1b3edbe2eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:40.740756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06000abb4d714d66a05dce1b3edbe2eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:40.747855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06000abb4d714d66a05dce1b3edbe2eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:40.819919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06000abb4d714d66a05dce1b3edbe2eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:40.820588 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06000abb4d714d66a05dce1b3edbe2eb'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:40.838080 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1390/2814] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:40 2022] DELETE /identity/v3/users/06000abb4d714d66a05dce1b3edbe2eb => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:40.842662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:40.843181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:40.843282 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:40.862399 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:40.863598 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:40.922684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1425/2815] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:40.933413 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:40.934893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:40.991047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdf45d-31e5-4c6e-8bff-00ea6e9bda4b admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:40.994928 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:40.995526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:40.995855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:40.996127 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:41.005017 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:41.006595 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:41.015919 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1391/2816] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:41.405283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:41.406719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:41.420316 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e7899-6468-4650-8ff1-663a5b6f0404 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:41.426003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:41.426851 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:41.427210 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:41.427585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] PATH_INFO: `/v3/projects/e6852f7c61a54191b8ee0c3c9d97cb51` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:41.429144 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6852f7c61a54191b8ee0c3c9d97cb51)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:41.431570 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:41.551318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6852f7c61a54191b8ee0c3c9d97cb51', 'request_id': 'req-b0e067b1-3641-43cd-80fc-58bca434ccf7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.556870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6852f7c61a54191b8ee0c3c9d97cb51', 'request_id': 'req-b0e067b1-3641-43cd-80fc-58bca434ccf7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.562925 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6852f7c61a54191b8ee0c3c9d97cb51', 'request_id': 'req-b0e067b1-3641-43cd-80fc-58bca434ccf7'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.642411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] The token cache is being invalidate because project e6852f7c61a54191b8ee0c3c9d97cb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:41.642697 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.643305 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.644810 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1426/2817] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:41 2022] DELETE /identity/v3/projects/e6852f7c61a54191b8ee0c3c9d97cb51 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:41.648986 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:41.658975 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a429b-956a-4a51-a325-530559d5ca51 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:41.715669 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:41.716356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:41.716636 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:41.716907 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] PATH_INFO: `/v3/users/d44aaf96280a4c138efb0b57a026ece0` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:41.718015 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44aaf96280a4c138efb0b57a026ece0)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:41.720021 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:41.984246 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:41.985779 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44aaf96280a4c138efb0b57a026ece0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.986436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44aaf96280a4c138efb0b57a026ece0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:41.993916 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44aaf96280a4c138efb0b57a026ece0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.079810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44aaf96280a4c138efb0b57a026ece0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.080446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-7521858d-de33-4f44-8601-00b61ad086a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44aaf96280a4c138efb0b57a026ece0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.097962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1392/2818] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:41 2022] DELETE /identity/v3/users/d44aaf96280a4c138efb0b57a026ece0 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:42.530393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:42.531905 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:42.542251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:42.548754 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e067b1-3641-43cd-80fc-58bca434ccf7 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:42.606425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:42.607050 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:42.607325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:42.607594 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] PATH_INFO: `/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:42.608752 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed44be57a674a72a37e29cd53f4b9ef)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:42.610639 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:42.751546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed44be57a674a72a37e29cd53f4b9ef', 'request_id': 'req-a223b211-94b7-494a-a1e9-13e9f3b0a72d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.756977 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed44be57a674a72a37e29cd53f4b9ef', 'request_id': 'req-a223b211-94b7-494a-a1e9-13e9f3b0a72d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.763046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed44be57a674a72a37e29cd53f4b9ef', 'request_id': 'req-a223b211-94b7-494a-a1e9-13e9f3b0a72d'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.865501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] The token cache is being invalidate because project 5ed44be57a674a72a37e29cd53f4b9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:42.865784 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.869140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-a223b211-94b7-494a-a1e9-13e9f3b0a72d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:42.869140 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1427/2819] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:42 2022] DELETE /identity/v3/projects/5ed44be57a674a72a37e29cd53f4b9ef => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:42.873103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:42.873566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:42.873693 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:42.893668 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:42.894876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:42.957652 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1393/2820] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:42.965815 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:42.966100 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:42.966377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:42.985726 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:42.986911 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:42.996017 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1428/2821] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:42.999976 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.009637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83935c53-04ef-4ab6-94a8-9b5557295b8f None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.014159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.014745 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.015022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.015295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.015931 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:17:43.016265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.017864 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.029273 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1394/2822] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:17:43.033458 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.043347 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40829513-eae3-4c4e-bbda-e8534c77af84 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.047942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.048577 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.048854 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.049125 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.049737 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.051656 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.189870 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1429/2823] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:43 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:43.194066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.204129 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e18f9-e021-46a6-8044-87f175e42513 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.260476 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.261078 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.261358 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.261635 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.262265 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.264224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.374043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1395/2824] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:43 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:43.379693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.390300 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1d29-5c8f-4dc8-9b29-5751f96387f0 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.394850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.395470 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.395753 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.396720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.397627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.399219 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.408943 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1430/2825] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:43.413585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.423615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a5833-4dfb-4200-ab8b-f520c4e2bd52 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.428475 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.429065 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.429340 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.429607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.430239 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.431897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b580-64cb-4f2e-b1dc-8f49e6060d15 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.442488 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1396/2826] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:17:43.446591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.456567 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86a185-1ca3-41e4-909b-03cc3dc17d0a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.461950 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.462616 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.462892 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.463163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] PATH_INFO: `/v3/projects/95f2e304e2144a5e950d2deee5e58585/users/ce7f12702ee846a6a52bf91608520e16/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.478230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f2e304e2144a5e950d2deee5e58585, user_id=ce7f12702ee846a6a52bf91608520e16, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.481044 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.568740 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1431/2827] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:43 2022] PUT /identity/v3/projects/95f2e304e2144a5e950d2deee5e58585/users/ce7f12702ee846a6a52bf91608520e16/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:43.579125 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.579417 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.579697 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.608621 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] MFA Rules not processed for user `ce7f12702ee846a6a52bf91608520e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:43.609875 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:43.629212 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:43.676610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1397/2828] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:43.687351 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:43.689055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:43.725334 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:43.748622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c31c-6bf3-4dae-9400-516f90f092ed admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:43.752283 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57f08a89a0dd4befb153c2b38167b5c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:43.752883 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:43.753163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:43.753435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:43.763828 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:43.765286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54339de-c4fd-4eb3-8dee-5a198bb43463 service cinder] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:43.775346 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1432/2829] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:17:53.734266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=19 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:53.842856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=12 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:55.390444 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:55.453310 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721daba6-1b50-4cd3-a5a3-c222565abaed None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:55.457626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:55.458865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:55.459141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:55.459409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] PATH_INFO: `/v3/users/ce7f12702ee846a6a52bf91608520e16` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:55.460559 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7f12702ee846a6a52bf91608520e16)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:55.462462 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:55.659819 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7f12702ee846a6a52bf91608520e16'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:55.660480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7f12702ee846a6a52bf91608520e16'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:55.667492 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7f12702ee846a6a52bf91608520e16'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:55.748289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7f12702ee846a6a52bf91608520e16'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:55.748914 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7f12702ee846a6a52bf91608520e16'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:55.767416 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1398/2830] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:17:55 2022] DELETE /identity/v3/users/ce7f12702ee846a6a52bf91608520e16 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 25 03:17:55.772478 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:55.772951 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:55.773080 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:55.791574 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:55.792858 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:55.850959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1433/2831] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:55.861233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:55.862663 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:55.923594 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9a41f-a6fe-4428-811c-7932b6073618 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:55.927617 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:55.928427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:55.928707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:55.928978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:55.938290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:55.939718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc219b25-fbb2-486b-a663-4bb4399afbf5 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:55.949533 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1399/2832] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:17:56.356256 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:56.357778 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:56.373747 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a40dd-d49f-43f4-b759-ac4fcd04d7c9 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:56.378459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:56.379099 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.380589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.380882 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] PATH_INFO: `/v3/projects/95f2e304e2144a5e950d2deee5e58585` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.382259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f2e304e2144a5e950d2deee5e58585)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:56.384933 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:56.474684 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f2e304e2144a5e950d2deee5e58585', 'request_id': 'req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:56.480336 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f2e304e2144a5e950d2deee5e58585', 'request_id': 'req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:56.486625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f2e304e2144a5e950d2deee5e58585', 'request_id': 'req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:56.561510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] The token cache is being invalidate because project 95f2e304e2144a5e950d2deee5e58585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:17:56.561797 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:56.562407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-e4cbafb3-5d04-4c3b-9fff-ed9d3e72b8f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:17:56.563871 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1434/2833] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:17:56 2022] DELETE /identity/v3/projects/95f2e304e2144a5e950d2deee5e58585 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:56.569019 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.569528 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.569646 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.588192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:56.589377 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:56.606960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:56.649228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1400/2834] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:17:56.656910 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.657195 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.657466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.678372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:56.679641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:56.687711 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1435/2835] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:56.691868 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:56.701590 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08fb4e-7f1f-4997-8ed3-b4bb2eacca22 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:56.706036 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:56.706660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.706934 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.707205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.707839 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:17:56.708322 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:56.709946 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:56.721407 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1401/2836] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:17:56.725960 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:56.735327 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238227e-d743-43a7-9de3-7d6633d00390 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:56.740526 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:56.741380 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.741655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.741923 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.742551 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:56.744513 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:56.860723 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1436/2837] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:17:56 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:17:56.864731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:56.875048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374676f9-4c8e-4442-90cd-fc8fedbba8ac admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:56.931484 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:56.932095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:56.932403 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:56.932660 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:56.933306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:56.935262 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:57.046196 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1402/2838] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:17:56 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:17:57.061964 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:57.074764 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcc094-a7bd-46c0-ad7e-8ae96abb9254 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:57.079289 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:57.079954 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:57.080278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:57.080662 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:57.081678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:57.084648 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:57.094641 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1437/2839] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:57.099255 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:57.109529 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb5df5-ce33-4315-8940-3446e56dcfd4 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:57.114118 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:57.115043 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:57.115339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:57.115612 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:57.116567 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:57.118159 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e8f46-1e42-47a2-9ce5-dcdb5efb0548 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:57.128446 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1403/2840] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:17:57.133279 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:57.143056 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d23b2-8b6e-44b6-a2e2-888064458f7e admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:57.148332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:57.148965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:57.149239 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:57.149510 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] PATH_INFO: `/v3/projects/1e9654067eab44baaac7631bf9e2fc9a/users/bcff78c2fee849979692402ae298cbd3/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:57.163961 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9654067eab44baaac7631bf9e2fc9a, user_id=bcff78c2fee849979692402ae298cbd3, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:57.166606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:57.268369 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1438/2841] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:17:57 2022] PUT /identity/v3/projects/1e9654067eab44baaac7631bf9e2fc9a/users/bcff78c2fee849979692402ae298cbd3/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:17:57.274464 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e137fb-1835-4f28-bce7-7ad4a4e07aa9 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:57.274817 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e137fb-1835-4f28-bce7-7ad4a4e07aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:57.275154 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e137fb-1835-4f28-bce7-7ad4a4e07aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:57.294165 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-05e137fb-1835-4f28-bce7-7ad4a4e07aa9 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:17:57.295372 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-05e137fb-1835-4f28-bce7-7ad4a4e07aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:57.352542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1404/2842] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:17:57.361928 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:17:57.363712 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:17:57.386918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:17:57.423151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfdacd-7118-46b0-989e-3807d6325eef admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:17:57.426953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:17:57.427558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:17:57.427909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:17:57.428177 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:17:57.437320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:17:57.438900 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:17:57.449856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1439/2843] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:02.893349 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:02.893869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:02.893869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:02.952325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] MFA Rules not processed for user `bcff78c2fee849979692402ae298cbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:02.953721 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:02.975290 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:03.035915 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1405/2844] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:18:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:03.040259 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.051216 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf237-d843-43d5-b265-77c7c1bef125 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.056728 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.057377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.057651 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.057919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.058589 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.060595 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.187260 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1440/2845] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:18:03 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:18:03.187479 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.201543 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727913e-3c42-4810-aa1e-b90f39517615 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.274582 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.275201 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.275483 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.275765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.276455 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.278389 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.375968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1406/2846] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:18:03 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 25 03:18:03.377160 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.387953 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c4bf6-be52-4767-952e-5c1517b6e89f admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.392476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.393082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.393359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.393629 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.394271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.395942 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.407640 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1441/2847] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:18:03.412104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.423041 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a8fc-2fd6-4b08-abad-88dd415a3485 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.428047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.429569 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.429869 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.430144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] PATH_INFO: `/v3/projects/1d2e7465e9854c168f0f10b0675afc2c/users/4ed71304785f460296637f3c10efdeb9/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.445227 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2e7465e9854c168f0f10b0675afc2c, user_id=4ed71304785f460296637f3c10efdeb9, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.447906 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.514738 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1407/2848] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:03 2022] PUT /identity/v3/projects/1d2e7465e9854c168f0f10b0675afc2c/users/4ed71304785f460296637f3c10efdeb9/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:03.518649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.529600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090030a0-5389-4b51-b889-bae67b093548 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.588248 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.588955 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.589241 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.589515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.590152 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.591725 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92691c5f-0137-4ec4-987c-7690a7e690b3 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.602264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1442/2849] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:18:03.606502 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:03.618187 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056ad9-d537-4656-a36b-5fb205efa5e6 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:03.622930 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:03.623596 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:03.623876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:03.624169 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] PATH_INFO: `/v3/projects/1d2e7465e9854c168f0f10b0675afc2c/users/4ed71304785f460296637f3c10efdeb9/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:03.626103 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2e7465e9854c168f0f10b0675afc2c, user_id=4ed71304785f460296637f3c10efdeb9, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:03.628997 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:03.641013 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1408/2850] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:18:03 2022] PUT /identity/v3/projects/1d2e7465e9854c168f0f10b0675afc2c/users/4ed71304785f460296637f3c10efdeb9/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:18:09.145989 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eea9a-b6db-4514-8fe9-af0ef3599b4f None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:09.146469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eea9a-b6db-4514-8fe9-af0ef3599b4f None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:09.146587 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eea9a-b6db-4514-8fe9-af0ef3599b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:09.180683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-e03eea9a-b6db-4514-8fe9-af0ef3599b4f None None] MFA Rules not processed for user `4ed71304785f460296637f3c10efdeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:09.182064 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e03eea9a-b6db-4514-8fe9-af0ef3599b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:09.272602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1443/2851] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:18:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:09.285709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:09.289147 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:09.394401 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea04127-2b12-4860-8a69-e8350d22366d admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:09.399456 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca96b68db5ab4c42ae4d34f6ce0a21d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:09.400071 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:09.400409 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:09.400680 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:09.405224 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:09.465151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:09.466774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:09.477084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1409/2852] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:18:09.732363 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42bfe-20a9-402b-bc8f-bd16cdd0721b None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:09.732650 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42bfe-20a9-402b-bc8f-bd16cdd0721b None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:09.732948 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42bfe-20a9-402b-bc8f-bd16cdd0721b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:09.759903 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:09.786957 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3bb42bfe-20a9-402b-bc8f-bd16cdd0721b None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:09.788264 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3bb42bfe-20a9-402b-bc8f-bd16cdd0721b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:09.855791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1444/2853] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:18:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:10.044749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:10.046341 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:10.140774 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3694-b72c-44e2-8b80-90df08bd0049 service glance] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:10.145782 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:10.146384 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:10.146666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:10.147141 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:10.155180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:10.156810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:10.165412 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1410/2854] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:18:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:18:19.908839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=35 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:20.161480 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=66 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:20.458331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:20.458685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:20.459078 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:20.479652 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] MFA Rules not processed for user `bcff78c2fee849979692402ae298cbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:20.480919 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:20.487807 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1445/2855] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:18:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:20.752253 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:20.807666 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ec15-af79-43f7-9bed-0bf3910b16b8 service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:20.811295 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:20.811899 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:20.812199 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:20.812500 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:20.814210 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:20.824072 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:20.825520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80122800-c937-4b88-9bb3-e26a7102e11a service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:20.834442 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1411/2856] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:21.255215 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:21.257278 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:21.330124 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d2041-adc3-484e-8dc0-1da9c00489b2 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:21.335907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:21.338521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:21.338521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:21.338521 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:21.338840 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:21.350799 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:21.352455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:21.362625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1446/2857] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:24.059163 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:24.059682 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:24.059851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:24.079523 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:18:24.080893 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:24.088962 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1412/2858] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Aug 25 03:18:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:18:24.102918 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:24.114407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd907-20bf-47ee-8d1e-8f58aaf882f3 service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:24.120302 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:24.120907 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:24.121183 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:24.121452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:24.129695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:24.131185 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:24.139343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1447/2859] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:18:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:18:32.422040 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:32.434192 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3aaea-44b9-4e01-bcfb-5ec8c4547a1a None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:32.439978 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:32.441035 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:32.441524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:32.441903 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:32.445123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:32.456134 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:32.458355 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:32.470256 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1413/2860] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:34.136133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=41 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:39.433164 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:39.435415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:39.507959 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001fcbc-3098-4059-8732-6b5a0a5cba3b service swift] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:39.513455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:39.514059 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:39.514338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:39.514606 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:39.516328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:39.529203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:39.540972 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:39.542608 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1448/2861] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:18:42.465764 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=35 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:49.538515 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=34 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:18:53.412961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:18:53.414426 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:18:53.425334 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620068f6-9cc8-4e2f-8ffd-f1feaa181694 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:18:53.428980 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:18:53.429579 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:18:53.429855 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:18:53.430123 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:18:53.439904 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:18:53.441364 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:18:53.451190 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1414/2862] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:18:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:03.446536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:23.370313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:23.371783 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:23.386011 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a19caf-8749-4725-8aba-d1c7f4427cc4 service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:23.391136 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:23.392034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:23.392415 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:23.392691 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:23.394467 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:23.406508 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:23.408996 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601656c-daa1-43ec-8883-84999a6735e3 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:23.419497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1449/2863] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:19:33.415544 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=6 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:19:58.566988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:19:58.584960 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0608-9c28-4a30-89b0-97f88008bbef service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:19:58.596605 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:19:58.598202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:19:58.598552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:19:58.599066 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:19:58.601144 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:19:58.629042 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:19:58.630776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:19:58.644404 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1415/2864] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:19:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:07.873730 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:07.874204 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:07.874309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:07.893372 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] MFA Rules not processed for user `66f04604a2f44191a8af25bab59e44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:07.894536 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:07.901476 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1450/2865] 10.222.0.75 () {58 vars in 967 bytes} [Thu Aug 25 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:07.943339 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:07.955162 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a07605-ede9-4d94-bad0-96f32eeffc34 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:07.960020 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8798813bd1e461d81faf73696f62a16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:07.960632 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:07.960909 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:07.961177 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:07.969126 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:07.970849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:07.978793 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1416/2866] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Aug 25 03:20:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 25 03:20:12.370366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:12.383995 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7417b6-64b4-41ca-8fbd-4038ddf7bc15 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:12.389150 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:12.389917 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:12.390235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:12.391071 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:12.392801 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:12.404757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:12.407005 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:12.417540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1451/2867] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:16.152754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.217094 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58953e9-8c31-418b-a527-cbd8b81dff1d service swift] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.222000 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.222874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.223157 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.223424 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] PATH_INFO: `/v3/users/bcff78c2fee849979692402ae298cbd3` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.224662 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcff78c2fee849979692402ae298cbd3)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.226691 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.387865 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcff78c2fee849979692402ae298cbd3'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.388539 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcff78c2fee849979692402ae298cbd3'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.395398 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcff78c2fee849979692402ae298cbd3'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.454591 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcff78c2fee849979692402ae298cbd3'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.455161 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcff78c2fee849979692402ae298cbd3'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.472801 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1417/2868] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:16 2022] DELETE /identity/v3/users/bcff78c2fee849979692402ae298cbd3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:16.731139 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:16.743286 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57ae54-7c5e-4403-afab-a20956e4c436 service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:16.801930 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:16.802563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:16.802842 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:16.803114 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] PATH_INFO: `/v3/projects/1e9654067eab44baaac7631bf9e2fc9a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:16.804274 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9654067eab44baaac7631bf9e2fc9a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:16.806193 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:16.935649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9654067eab44baaac7631bf9e2fc9a', 'request_id': 'req-46636c4e-755e-45ef-91c8-0a87cf30cc35'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.941477 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9654067eab44baaac7631bf9e2fc9a', 'request_id': 'req-46636c4e-755e-45ef-91c8-0a87cf30cc35'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:16.948158 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9654067eab44baaac7631bf9e2fc9a', 'request_id': 'req-46636c4e-755e-45ef-91c8-0a87cf30cc35'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.020003 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] The token cache is being invalidate because project 1e9654067eab44baaac7631bf9e2fc9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:17.020226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.020867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.022393 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1452/2869] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:16 2022] DELETE /identity/v3/projects/1e9654067eab44baaac7631bf9e2fc9a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.026421 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.027862 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.040824 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd08b9-fc5f-4592-99ce-6c29b92a4062 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.088452 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:17.097108 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.100306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.100306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.100306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] PATH_INFO: `/v3/users/4ed71304785f460296637f3c10efdeb9` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.100306 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed71304785f460296637f3c10efdeb9)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.101371 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.262810 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed71304785f460296637f3c10efdeb9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.268084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed71304785f460296637f3c10efdeb9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.271003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed71304785f460296637f3c10efdeb9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.346137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed71304785f460296637f3c10efdeb9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.347829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-570bea98-49b6-47a9-9d90-ec0df2153690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed71304785f460296637f3c10efdeb9'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.367010 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1418/2870] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:20:17 2022] DELETE /identity/v3/users/4ed71304785f460296637f3c10efdeb9 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.605328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:17.606758 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.621176 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636c4e-755e-45ef-91c8-0a87cf30cc35 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:17.680833 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:17.681563 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.681841 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.682115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] PATH_INFO: `/v3/projects/1d2e7465e9854c168f0f10b0675afc2c` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.683226 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2e7465e9854c168f0f10b0675afc2c)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:17.685220 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:17.696735 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:17.792773 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2e7465e9854c168f0f10b0675afc2c', 'request_id': 'req-ef40b795-22e1-46bf-818b-c777bd9f0ceb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.798151 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2e7465e9854c168f0f10b0675afc2c', 'request_id': 'req-ef40b795-22e1-46bf-818b-c777bd9f0ceb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.804170 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2e7465e9854c168f0f10b0675afc2c', 'request_id': 'req-ef40b795-22e1-46bf-818b-c777bd9f0ceb'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.870946 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] The token cache is being invalidate because project 1d2e7465e9854c168f0f10b0675afc2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81785) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:20:17.871235 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.871856 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-ef40b795-22e1-46bf-818b-c777bd9f0ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:20:17.873358 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1453/2871] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:20:17 2022] DELETE /identity/v3/projects/1d2e7465e9854c168f0f10b0675afc2c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:17.878576 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.878866 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.879148 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.899189 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:17.900422 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.960781 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1419/2872] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:17.968558 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:17.968843 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:17.969121 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:17.989455 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:17.990649 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:17.998785 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1454/2873] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:18.003716 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.014028 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1237a-3a54-4376-a942-482a19250c79 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.019231 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.019849 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.020128 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.020431 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.021070 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81786) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 25 03:20:18.021376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.022977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.034707 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1420/2874] 10.222.0.75 () {58 vars in 1153 bytes} [Thu Aug 25 03:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 25 03:20:18.038855 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.047936 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa834dd-1a41-416e-866c-a1956aabc4db None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.052683 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.053275 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.053550 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.053823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.054435 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.056366 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.158720 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1455/2875] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:18.163625 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.173736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115a755-5a4c-419c-a2f4-6722fb112581 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.232047 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.232678 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.232961 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.233234 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.233854 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.235758 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.244498 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:18.352299 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1421/2876] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:18.356391 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.367313 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419412e1-c3c3-4c45-9e9d-adee573c9ea7 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.371940 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.372580 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.372867 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.373141 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.373847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.375426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.386181 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1456/2877] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:18.390091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.401599 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e23ee-eb09-44bd-b0a3-1afb1bec46fd admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.406158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.406749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.407030 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.407303 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.407942 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.409972 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb34411-2def-44bb-a39c-d4b3d550f1fd admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.419836 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1422/2878] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:18.423816 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.433880 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006c25f-15b7-444c-9ccd-a7e629284808 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.438421 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.439063 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] REQUEST_METHOD: `PUT` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.443452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.443452 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] PATH_INFO: `/v3/projects/0bb0777527214f059e031c3b484a2eb1/users/b841ba312a6e4b919719c3166248c149/roles/5baef496d0274b11922ddd2613e3ff1a` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.454814 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb0777527214f059e031c3b484a2eb1, user_id=b841ba312a6e4b919719c3166248c149, role_id=5baef496d0274b11922ddd2613e3ff1a)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.458333 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.552430 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1457/2879] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:18 2022] PUT /identity/v3/projects/0bb0777527214f059e031c3b484a2eb1/users/b841ba312a6e4b919719c3166248c149/roles/5baef496d0274b11922ddd2613e3ff1a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:18.557585 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1ce8d-5914-4e10-a5ed-aa54681b4002 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.557874 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1ce8d-5914-4e10-a5ed-aa54681b4002 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.558150 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1ce8d-5914-4e10-a5ed-aa54681b4002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.578751 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-56a1ce8d-5914-4e10-a5ed-aa54681b4002 None None] MFA Rules not processed for user `ba566fd127094ca6936cf4d2fd42c8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:18.579943 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-56a1ce8d-5914-4e10-a5ed-aa54681b4002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:18.639399 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1423/2880] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:18.649142 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:18.650839 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:18.706771 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae176e70-3bf3-4986-8273-d4e64874874b admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:18.710280 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:18.710879 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:18.711159 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:18.711433 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:18.720175 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:18.721714 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:18.730891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1458/2881] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:23.512537 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:23.513423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:23.513423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:23.545626 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] MFA Rules not processed for user `b841ba312a6e4b919719c3166248c149`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:23.546981 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:23.629892 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1424/2882] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:23.633767 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:23.644860 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d40d9-7f2d-4530-8c05-bd999b141e58 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:23.651524 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:23.652163 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:23.652449 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:23.652719 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:23.653501 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:23.655407 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:23.727837 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1459/2883] 10.222.0.75 () {60 vars in 1153 bytes} [Thu Aug 25 03:20:23 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:23.731898 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:23.745999 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1dd15-79df-4e49-8510-b31372b25b84 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:23.809552 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:23.810164 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:23.810441 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:23.811266 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] PATH_INFO: `/v3/users` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:23.812088 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:23.814054 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:23.958718 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:23.964858 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1425/2884] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Aug 25 03:20:23 2022] POST /identity/v3/users => generated 295 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 25 03:20:23.968901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:23.979838 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1657a1-8160-4434-b5bc-cf3ff338b789 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:23.984281 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:23.984874 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:23.985149 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:23.985418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:23.986049 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:23.988238 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:23.999822 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1460/2885] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 25 03:20:24.004257 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:24.015867 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28c57-4e77-4c1c-8f4a-41fde29f1645 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:24.021581 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:24.022243 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:24.022526 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:24.022802 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] PATH_INFO: `/v3/projects/9685fa9339a54ad39ff37a59c8e56108/users/4f89457f95fe4cde8231e8df48d0a111/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:24.039014 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9685fa9339a54ad39ff37a59c8e56108, user_id=4f89457f95fe4cde8231e8df48d0a111, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:24.041756 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:24.112084 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1426/2886] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:24 2022] PUT /identity/v3/projects/9685fa9339a54ad39ff37a59c8e56108/users/4f89457f95fe4cde8231e8df48d0a111/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:24.116685 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:24.126534 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998815f-e415-4968-90ec-819b4a58fd1d admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:24.130600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:24.180748 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:24.181350 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:24.181625 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:24.181893 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] PATH_INFO: `/v3/roles` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:24.182522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:24.184093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202b47-9559-4209-9175-ffa49cde278a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:24.194418 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1461/2887] 10.222.0.75 () {58 vars in 1122 bytes} [Thu Aug 25 03:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 25 03:20:24.198712 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:24.211185 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3454734-127f-4066-b553-97f5c91c2195 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:24.216032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:24.216726 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:24.217007 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:24.217284 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] PATH_INFO: `/v3/projects/9685fa9339a54ad39ff37a59c8e56108/users/4f89457f95fe4cde8231e8df48d0a111/roles/a6f0106bea874efe895b3a8ed472a48f` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:24.219223 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9685fa9339a54ad39ff37a59c8e56108, user_id=4f89457f95fe4cde8231e8df48d0a111, role_id=a6f0106bea874efe895b3a8ed472a48f)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:24.221924 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:24.233228 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1427/2888] 10.222.0.75 () {60 vars in 1483 bytes} [Thu Aug 25 03:20:24 2022] PUT /identity/v3/projects/9685fa9339a54ad39ff37a59c8e56108/users/4f89457f95fe4cde8231e8df48d0a111/roles/a6f0106bea874efe895b3a8ed472a48f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:20:29.924979 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae30e5a-304a-4c18-89b9-51ecca037879 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:29.925966 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae30e5a-304a-4c18-89b9-51ecca037879 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:29.926075 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae30e5a-304a-4c18-89b9-51ecca037879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:29.959133 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-aae30e5a-304a-4c18-89b9-51ecca037879 None None] MFA Rules not processed for user `4f89457f95fe4cde8231e8df48d0a111`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:29.960902 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-aae30e5a-304a-4c18-89b9-51ecca037879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:30.033522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1462/2889] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 25 03:20:30.056202 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:30.057988 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:30.141286 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de740-306a-4ff4-91c1-e5fe9167fd0c admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:30.147775 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:30.148445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:30.148744 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:30.148993 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:30.206099 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:30.208117 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:30.217197 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1428/2890] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:40.030426 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=60 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:40.213091 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:48.455682 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:48.456212 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:48.456343 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:48.474895 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] MFA Rules not processed for user `b841ba312a6e4b919719c3166248c149`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:20:48.476085 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:48.482824 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1463/2891] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:20:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:20:48.798847 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:20:48.852732 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38883af-3cf3-4cff-a77f-89c122d65567 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:20:48.856517 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:20:48.861524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:20:48.861524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:20:48.861524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:20:48.861524 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:20:48.869690 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:20:48.871109 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:20:48.879003 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1429/2892] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:20:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:20:58.469546 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:20:58.875485 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=26 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:09.054213 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:09.056035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:09.067823 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d9f6f-58a4-4975-bbdb-3082ce23597e None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:09.072325 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:09.072920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:09.073203 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:09.073471 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:09.075115 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:09.088655 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:09.090058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:09.100759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1464/2893] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:10.279205 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:10.287765 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90c5c1-e541-4466-a29f-569fb3ff2183 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:10.292248 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:10.292845 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:10.293151 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:10.293423 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:10.295114 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:21:10.308305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:10.309736 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:10.320587 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1430/2894] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:10.623672 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:21:10.632707 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b587b1-94b6-407f-8161-08fd7a8e537b service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:21:10.636530 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:21:10.637120 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:21:10.637394 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:21:10.637659 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:21:10.646312 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:21:10.647706 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858fa33-68ea-4211-abd7-2ae10fcd76b2 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:21:10.657061 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1465/2895] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:21:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:21:20.315749 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:21:20.652891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=12 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:10.763333 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:10.775790 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1730b1a-6473-476c-840c-71df17c6eb85 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:10.782092 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:10.782709 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:10.782990 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:10.783263 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:10.785124 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:10.798723 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:10.800356 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:10.813086 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1431/2896] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:12.667108 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] REQUEST_METHOD: `POST` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:12.667585 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:12.667759 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:12.687251 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.auth.core [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] MFA Rules not processed for user `b841ba312a6e4b919719c3166248c149`. Rule list: `[]` (Enabled: `True`). {{(pid=81785) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:23:12.688469 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:12.695201 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1466/2897] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:23:20.808048 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:22.682502 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=3 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:23:32.622876 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:23:32.632520 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614f90-ec22-4fb3-a0e5-f6fb3507acc5 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:23:32.641009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:23:32.641009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:23:32.641009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:23:32.641009 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:23:32.641593 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:23:32.650601 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:23:32.652137 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384b24b-44cb-4808-b917-48ea3ef29ce0 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:23:32.661305 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1432/2898] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:23:42.658211 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:24:29.341973 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:24:29.343420 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:29.509459 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f711d18-35ab-4d79-98fb-96f0529b26f0 None None] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:24:29.515695 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:24:29.516540 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:24:29.516821 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:24:29.517093 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:24:29.519676 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:24:29.533949 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:24:29.550265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:24:29.550265 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1467/2899] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:24:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:24:39.541899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=63 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:10.469226 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] REQUEST_METHOD: `POST` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:10.469833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:10.469833 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:10.489766 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.auth.core [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] MFA Rules not processed for user `b841ba312a6e4b919719c3166248c149`. Rule list: `[]` (Enabled: `True`). {{(pid=81786) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 25 03:25:10.491056 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:10.499471 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1433/2900] 10.222.0.75 () {58 vars in 958 bytes} [Thu Aug 25 03:25:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 25 03:25:10.839228 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:10.860425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191fdc-5c95-492b-b356-be18158d7f9d service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:10.864057 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:10.864774 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:10.865055 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:10.865329 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:10.867035 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:10.890696 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:10.892082 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:10.900465 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1468/2901] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:25:11.647206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:11.648657 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:11.664616 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03ec43-6b9d-4f16-bc5e-8eaa50d315d4 None None] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:11.671918 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:11.676002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:11.676002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:11.676002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:11.676002 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:11.686829 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:11.688521 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:11.703022 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1434/2902] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:25:20.896850 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=15 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:21.695233 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=9 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:25.534602 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:25.565599 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ebfbc-73be-420e-b679-fcec7a57ac21 service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:25.571425 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:25.572046 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:25.572359 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:25.572638 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:25.575332 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:25.602553 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:25.604813 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:25.638331 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1469/2903] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:25:30.452613 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:25:30.454120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:30.469851 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf0a0-a699-43f9-a350-31d55537e4ae service placement] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:25:30.475289 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867d21f5f80d4cc48cc53e0b89c7f2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:25:30.475910 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:25:30.476208 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:25:30.476473 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:25:30.484968 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:25:30.499615 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:25:30.501269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:25:30.512834 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1435/2904] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:25:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:25:35.614406 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=18 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:25:40.507792 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:01.085812 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:01.095591 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebfc3-c8e1-4393-ba5d-abe9dcd79e70 service nova] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:01.099693 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:01.100338 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:01.100619 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:01.100891 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:01.109249 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:01.110678 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:01.119690 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1470/2905] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:02.665509 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:02.676427 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3d245-feb2-40a8-aec6-df717f899584 service nova] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:02.680542 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:02.681268 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:02.681610 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:02.681951 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:02.694637 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:02.697095 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:02.715796 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1436/2906] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:27:03.917288 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:03.926698 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748116-a8a7-4448-82d0-378ed8bd456b service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:03.933413 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:03.934039 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:03.934320 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:03.934593 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:03.946963 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:03.948169 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:03.960205 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1471/2907] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:12.705761 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=6 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:13.954230 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=12 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:17.943687 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:17.952436 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926322b-8775-4ef2-9cee-85d85d92c4f8 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:17.956220 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c6b4345e6cd418f8b2a3b6e0b054ddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:17.956811 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:17.957089 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:17.957359 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:18.042376 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:18.043776 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:18.066696 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1437/2908] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 25 03:27:22.235432 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:22.252920 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37a06-1c8c-43c8-9ca8-4776c99039cb service neutron] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:22.258681 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:22.259328 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:22.259610 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:22.259909 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:22.262088 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:22.275381 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:22.277627 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:22.294484 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1472/2909] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:26.488731 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:26.500179 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92d570-61ff-4d75-9385-d7193181e310 service neutron] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:26.505269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:26.505897 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] REQUEST_METHOD: `GET` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:26.506174 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:26.506445 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:26.508335 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:26.521206 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:26.522923 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:26.534180 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1438/2910] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:27:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:27:26.610377 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:26.622601 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b57a2-a340-494b-85a7-d33154519e4e service placement] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:26.627556 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:26.628221 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:26.628520 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:26.628791 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] PATH_INFO: `/v3/users/b841ba312a6e4b919719c3166248c149` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:26.629901 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b841ba312a6e4b919719c3166248c149)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:26.631789 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:27.059497 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b841ba312a6e4b919719c3166248c149'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.060138 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b841ba312a6e4b919719c3166248c149'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.066899 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b841ba312a6e4b919719c3166248c149'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.207262 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b841ba312a6e4b919719c3166248c149'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.208271 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b841ba312a6e4b919719c3166248c149'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.232586 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1473/2911] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:27:26 2022] DELETE /identity/v3/users/b841ba312a6e4b919719c3166248c149 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:27.614917 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:27.626269 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75869-d5fd-444b-9768-2c32c53cb72f service placement] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:27.691158 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:27.691814 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:27.692120 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:27.692534 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] PATH_INFO: `/v3/projects/0bb0777527214f059e031c3b484a2eb1` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:27.693813 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb0777527214f059e031c3b484a2eb1)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:27.695822 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:27.988313 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb0777527214f059e031c3b484a2eb1', 'request_id': 'req-94cc95a3-c3a7-4371-b654-93aaedc248b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:27.993947 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb0777527214f059e031c3b484a2eb1', 'request_id': 'req-94cc95a3-c3a7-4371-b654-93aaedc248b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.181006 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb0777527214f059e031c3b484a2eb1', 'request_id': 'req-94cc95a3-c3a7-4371-b654-93aaedc248b0'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.182546 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:28.208032 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] The token cache is being invalidate because project 0bb0777527214f059e031c3b484a2eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:27:28.213411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.213411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.213411 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1439/2912] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:27:27 2022] DELETE /identity/v3/projects/0bb0777527214f059e031c3b484a2eb1 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:28.214965 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:28.216522 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:28.230622 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9269-1520-44c2-9d06-35b99cb417b5 admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:28.290129 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:28.290757 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:28.291036 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:28.291308 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] PATH_INFO: `/v3/users/4f89457f95fe4cde8231e8df48d0a111` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:28.292466 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f89457f95fe4cde8231e8df48d0a111)` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:28.294364 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:28.613058 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f89457f95fe4cde8231e8df48d0a111'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.613847 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f89457f95fe4cde8231e8df48d0a111'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.621309 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f89457f95fe4cde8231e8df48d0a111'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.625243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:28.784004 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f89457f95fe4cde8231e8df48d0a111'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.784664 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.notifications [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f89457f95fe4cde8231e8df48d0a111'} {{(pid=81785) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:28.803439 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1474/2913] 10.222.0.75 () {58 vars in 1224 bytes} [Thu Aug 25 03:27:28 2022] DELETE /identity/v3/users/4f89457f95fe4cde8231e8df48d0a111 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:29.111292 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Authenticating user token {{(pid=81786) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:27:29.121643 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.fernet_utils [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81786) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:27:29.126977 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc95a3-c3a7-4371-b654-93aaedc248b0 admin admin] Validating token access rules against request {{(pid=81786) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:27:29.187193 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba566fd127094ca6936cf4d2fd42c8e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eabb40ce2ddb40b4bea318792fc09f8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81786) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:27:29.187754 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:27:29.188034 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:27:29.188337 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] PATH_INFO: `/v3/projects/9685fa9339a54ad39ff37a59c8e56108` {{(pid=81786) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:27:29.189566 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9685fa9339a54ad39ff37a59c8e56108)` {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:27:29.191478 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] RBAC: Authorization granted {{(pid=81786) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:27:29.432973 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9685fa9339a54ad39ff37a59c8e56108', 'request_id': 'req-cb660123-72e8-407a-9d28-6a8561b80db4'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:29.438325 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9685fa9339a54ad39ff37a59c8e56108', 'request_id': 'req-cb660123-72e8-407a-9d28-6a8561b80db4'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:29.523288 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9685fa9339a54ad39ff37a59c8e56108', 'request_id': 'req-cb660123-72e8-407a-9d28-6a8561b80db4'} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:29.536607 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] The token cache is being invalidate because project 9685fa9339a54ad39ff37a59c8e56108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81786) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 25 03:27:29.536900 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:29.537536 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG keystone.notifications [None req-cb660123-72e8-407a-9d28-6a8561b80db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81786) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 25 03:27:29.539104 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: [pid: 81786|app: 0|req: 1440/2914] 10.222.0.75 () {58 vars in 1233 bytes} [Thu Aug 25 03:27:29 2022] DELETE /identity/v3/projects/9685fa9339a54ad39ff37a59c8e56108 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 25 03:27:39.532553 dv-cinder-779578-6-iscsi devstack@keystone.service[81786]: DEBUG dbcounter [-] [81786] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81786) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:27:39.615045 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 25 03:29:34.306600 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Authenticating user token {{(pid=81785) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 25 03:29:34.308034 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:34.380236 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ecaa-f4bd-463e-8ce3-4f71e1f6754a admin admin] Validating token access rules against request {{(pid=81785) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 25 03:29:34.439463 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8912c5edc2964df1afb96fd23f02e299', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932be52c2fb94ef7bad55913e4e24363', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81785) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 25 03:29:34.440070 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] REQUEST_METHOD: `GET` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 25 03:29:34.440368 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] SCRIPT_NAME: `/identity` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 25 03:29:34.440643 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81785) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 25 03:29:34.442411 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.fernet_utils [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81785) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 25 03:29:34.454645 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 25 03:29:34.456243 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696b527-b7a2-479b-9186-c03179831a5a service placement] RBAC: Authorization granted {{(pid=81785) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 25 03:29:34.466318 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: [pid: 81785|app: 0|req: 1475/2915] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Aug 25 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 25 03:29:44.461832 dv-cinder-779578-6-iscsi devstack@keystone.service[81785]: DEBUG dbcounter [-] [81785] Writing DB stats keystone:SELECT=58 {{(pid=81785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}